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CHAPTER I 

INTRODUCTION 

The Purpose of Tliis Study 

The purpose of this study was to determine whether the experience and quality 
of writing changes when novice writers who compose on computers are supported by 
a more interactive guided-writing program. Although many such computer programs 
are currently available, little evidence exists that this computerized support is 
"effective/' This study defined and tested three types of computer guidance designed 
to develop expert characteristics in novice writers. 

Expression of the Problem 

The research question addressed is: among novice writers, does using 
computer-guided writing software produce characteristics that are typical of more 
expert writers? These characteristics were measured by ratings of rhetorical 
effectiveness, awareness of the writing process, and degree of revision. 

Establishing a Need for the Current Study 

Extensive research has been pursued to better understand what goes on in the 
mind of a writer. Seminal and continuing work by Flower and Hayes (1981) and 
Bereiter and Scardamalia (1987) has been complemented with the ongoing research 
of many others. By the early 1980s, Flower and Hayes had described the act of 
composition as a set of interrelated subprocesses (1981). Planning, translating, 
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reviewing, and nn.onitoring (now more commonly referred to as planning, drafting, 
and revising) were identified as the cognitive tools used by writers for the variety of 
subtasks that comprise the act of writing. Although criticism of the cognitive model 
of writing exists, one of its detractors has called the work of Flower and Hayes 
"powerful and effective" (Elbow, in Brand, 1989, p. xiii). Consequently, the model 
has become a dominant strand in the research of composition and provides a basis 
upon which to elaborate. 

Some of these elaborations have rikeady identified predictable characteristics 
and strategies of both novice t ad expert writers (Bereiter & Scardamalia, 1987; 
Flower & Hayes, 1981; McCutchen, 1988). Consequently, instructional practices 
promoting more mature strategies within novice writers have been developed, most 
particularly by Bereiter and ScardamaUa. Thek instructional strategy known as 
procedural facilitation (providing novices with cognitive support for more complex 
executive strategies in the form of prompts or cues) has been shown to promote 
advanced cognition (1984b). Bereiter and Scardamalia offered a postscript outlining 
their initial attempts at integrating procedural facilitation with word processing. 
Thus, even though a wealth of descriptive data exist to inform the design of 
procedural facilitation, little research exists that defines a model for applying these 
strategies to a computerized guided-writing environment 

Interestingly, the influence of both cognitive process theory and 
microcomputers expanded in the early 1980s. Since then, teachers, researchers, and 
software developers have created programs coupling new understanding of the 
composing process to the power of computers in order to create new writing tools. 
An array of computer-assisted writing programs have been created by researchers, 
contributing such programs as Writers' Helper 11 (Wresch, 1984), QUILL (Rubin & 
Bruce, 1984), DRAFT (Neuwirth, 1984), Catch (Daiute, 1984), and WAND AH (Von 



3 

Blum & Cohen, 1984). Even though details of these programs vary, they all share a 
grounding in the writing process approach and meet the criterion put forth by 
Montague in her book Computers, Cognition and Writing Instruction (1990), that 
good interactive writing programs guide the student cognitively while in the process 
of writing. This cognitive guiding is achieved through prompting, instructing, 
provoking, and encouraging novice writers to expand their writing strategies and 
skills. Commercial computer-assisted writing programs, then, have implemented 
Bereiter and Scardamalia's notion of procedural facilitation. In addition, they have 
attempted to promote the global, problem-solving approach found in more expert 
writing strategies. However, as effective as computer-assisted writing programs 
sound, few assessments presently exist (Bonk & Reynolds, 1990). In fact, even as 
Bonk and Reynolds attempted to develop and test a theory of prompted writing, the 
lack of statistically significant findings forced them to admit that "evidence 
supporting the usefulness of thinking skill prompts within either a particular paper or 
over time was minimal" (p. 49). 

This lack of research on computer-guided support for writing is surprising 
because of the computer's recent ubiquity. Since the late 1970s, computers have 
increasmgly permeated our society and schools. Montague (1990) cited research 
demonstrating this fact: by 1986 over 2 million computers had found their way into 
American schools, a 25 percent increase from the previous year (Goodspeed^ in 
Montague). Thus, the question is no longer, "Will computers have an impact on 
schools in the future?" but "What will we do with our computers today?" Lillian 
Bridwell (1985) emphasizes the need to maximize the available computer resources 
when she writes, "instruction in the new medium was not keeping up with research on 
writing and the development of writing abilities" (p. 115). 
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The research on writing and the increased availability of computers offer an 
ideal opportunity for models of instructional design. Instructional design (ID) earned 
its place in academia and business during and after World War n because of its 
successes in aerospace and the military (Eraut, 1989). ID uses a systems approach 
that examines the facets involved in a complex task in order to create effective 
instruction. Specifically, the systems approach assesses the task and then designs and 
implements instruction focused upon the acquisition of specific competencies 
(UUmer, 1973). Because facilitating measurable learning is the goal, an evaluation 
mechanism stands as an integral part of an instructional system. Recent 
developments in learning theories such as cognitivism and constructivism (Dick, 
1991) have prompted instructional designers to increase the sophistication and 
subtlety of their models, and, therefore, the complexity of the tasks that might be 
evaluated. Thus, reliance on overt, behaviorist outcomes that tend toward 
reductionism has given way to appreciation of the covert cognition and feelings 
involved in learning complex tasks. Because designers now possess models to 
address the subtleties of cognition, they may now fruitfully join those pursuing the 
study of composition. With its purposeful approach an instructional design model 
may be able to prescribe learning experiences that promote a more expert level in the 
learner. This is particularly valid considering that descriptors exist for both novice 
and expert writers, thus offering the basis for building instruction. In summary, it is 
an auspicious time to apply the practices of instructional design to the theories offered 
by the research of writing. 
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The following questions were addressed by this study: 
L Does computer-guided writing software help students produce more 
rhetorically-effective compositions? 

2. Are there definable types of computer guides that promote specific expert- 
like performances in novices? 

3. What is the interaction between learner ability and computer support? 

4. Is there a relationship between computer support and a student's level of 
satisfaction in using computers as a writing tool? 

As shown in Figure 1, "Computer Guiding" represents the independent 
variable for each of the research questions. It is depicted as a force actively impacting 
the writer because computer-guided writing programs interrupt the novice's typical 
writing strategy. The research questions develop out of this impact and are discussed 
singly. 

Question #1 addresses a basic goal of computer-guided writing software: to be 
valuable, it should contribute to "better writing." Because no definitive evidence 
exists supporting this goal, the first question investigate^ whether computer-guided 
writing software affects a novice's ability to write with rhetorical effectiveness. 

Question #2 investigates the influence different types of computer guides have 
on a number of learner performances which serve as dependent variables. Research 
has indicated that the characteristics of novice writers contrast sharply with those of 
experts (Bereiter & Scardamalia, 1984a, 1984b, 1987; Flower <& Hayes 1981a, 1981b, 
1981c; McCutchen, 1988). Analysis of the differences between novice and expert 
writers suggests several types of prompts or guides that could promote more 
advanced strategies. In order to define computer-guided writing software, its various 
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Research Question #4 
User satisfaction 

Rhetorical 
effectiveness 

Research Question #1 

Expert 
characteristics 

Research Question #2 
Figure 1 

Relationships Examined in This Study 

features must also be described and their functions illuminated. The next chapter 
begins to establish a lexicon of guide-types that question #2 attempts to measure. 

Question #3 probes the interaction between computer guiding and learner 
ability. Because novice and expert writers differ in their strategies and performances, 
one might expect the influence of computer support to vary with ability level. 

Question #4 looks at the affective impact of a computer-guideu writing 
environment on the dependent variable of user-satisfaction. Because novice writers 
have limited cognitive resources (Bereiter & Scardamalia, 1987), a guiding 
environment must not be fmstrating to use. Also, because some aspects of a guided- 
writing program may be optional, the computer environment must be satisfying, or 
risk going unused. 
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CHAPTER II 
REVIEW OF THE LITERATURE 

Chapter Outline 

This chapter begins with a review of the research in writing, giving particular 
attention to the different cognitive strategies and behaviors of novice and expert 
writers. Next, it surveys commercially available computer-assisted writing programs. 
Then, it defines and proposes types of computer guides. The chapter concludes with 
a list of the hypotheses tested in this study. 

Writing Research 

A Brief Summary of Compositional Trends 

Trends in composition and rhetoric have reflected the intellectual and social 
milieu. Developments since the 1960s illustrate this point Typical of the early years, 
Jerome Bruner's The Process of Education (1960) suggested a pedagogy of 
composition that moved away from an emphasis on product to the exploration of 
writing as a process. Initial models depicted a linear three-stage process (Murray, 
1968) of pre- writing, writing, and re- writing. New approaches to the teaching of 
writing followed in attempts to define continually more accurate models. A variety of 
activities, strategies, and theories grew in this atmosphere. Behaviorists like Zoellner 
(1969) analyzed the "scribal act'' and sought ways to reinforce desirable writing 
*'acts.'' Young, Becker, and Pike, (1970) advocated a heuristic of discovery that 
emphasized the notion that the writer was an intricate and inextricable element in his 
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or her discourse community. Emphasis on such pre-writing activities as free writing 
and journals contributed to a self-expressionistic model of writing which received 
further encouragement from the student-centered focus of educational reforms and 
foment of the *60s. Such expressionist proponents as Elbow (1973), Macrorie (1970), 
and Murray (1968) viewed writing as an art that a student could learn, but that an 
instructor could not teach (Berlin, 1987). They saw writing as a method of discovery 
and self-actualization. Consequentiy, writing workshops took on the atmosphere of 
''Rogerian encounter groups" (Berlin, p. 148). ''Writing'' had demonstrably shifted 
away from a product separate from the author to being a tool for finding an "inner 
vision'' (Berlin, p. 153). Neo- Aristotelians, led by Edward Corbett, sought to re- 
establish rhetoric's Classical origins. In his article, "The Rhetoric of the Open Hand 
and the Rhetoric of the Closed Fist" (1969), Corbett argued that the excesses of 
expressionistic non-rationalism were contrary to the nature of persuasion, discussion, 
and, ultimately, discourse. 

Into this setting another major influence shifted the debate from the politics of 
composition to first-hand observation and new theories of learning. Janet Emig 
(1971) performed case studies on the writing processes of 12th grade students from 
which she drew new insights about the teaching of writing. First, she noticed that the 
students did not write in linear "stages" as the then current incarnation of the model 
suggested. Rather, they moved recursively among stages, suggesting a more 
complex, less systematic, aspect to composition. 

Similar work by Janice Lauer (1970) introduced the theories of cognitive 
psychology to the study of the writing process. Lauer shared the linguists' and 
expressionists' emphasis on discovery as the heart of writing, but she advocated a 
creative, problem-solving model that used heuristics as "flexible guides to effective 
guessing" (in Beriin, 1985, p. 161). Lauer's application of cognitive psychology to 
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the composing process was shared and developed by many, among them, Bereiter and 
Scardamalia (1984a, 1984b, 1985), Flower and Hayes (1981a & 1981b), and 
Matsuhashi(1981). 

The Cognitive Process Model 

Led by Flower and Hayes and their colleagues (Flower, 1981; Flower & 
Hayes, 1981a, 1981b; and Flower, L., Shriver, K., Carey, L., Haas, C, & Hayes, J., 
1989), the cognitive psychology of writing quickly established itself as a powerful 
descriptive model. These researchers divided the cognitivistic interpretation into 
three main structures: the writing task and environment; schema theory and long-term 
memory; and the writing process itself. This latter highlighted the heuristic, problem- 
solving aspect advocated by Lauer, as well as identified an executive monitor that 
prompts the recursive use of the subprocesses of planning, translating, and reviewing. 
Flower accented that although the process resembles a 1-2-3 approach, the 
subprocesses actually interact in a non-linear manner, ' 'where writers constantly 
retum to earlier steps such as planning in order to carry out later ones" (Rower, 1981, 
p. 50). Planning, drafting, and revising became the manuals, wrenches, and calipers 
that Flower and Hayes saw as instruments in a "tool kit" supplying a variety of 
cognitive tools for a variety of tasks (1981a). In addition, because writing presents an 
*'ill-structured" problem of integrating content and presentation, a problem-solving 
approach must be employed (Bereiter & Scardamalia, 1987). Through research by 
these investigators and others, a descriptive model of writing as a process of 
interrelated, recursive, and problem- solving subskills was firmly established. 

This approach has been criticized for its failure to take into account feelings as 
well as the entire affective nature of creativity. Alice Brand (1989), in particular, has 
argued that, "As a definitive paradigm for writing. the cognitive process model is 
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premature" (p. 35). Brand makes a persuasive case against the cognitivists' isolation 
and disregard for the interplay between the conscious and the unconscious, knowing 
and feeling. As the previous summar>' indicates, descriptive models of the writing 
process are highly charged with political, social, and philosophical currents. 
Certainly it is beyond the scope of this study to resolve the differences. And yet some 
theory must serve as the basis for any instructional design. Because of its 
applicability to the systems approach of ID, not to mention a body of supportive 
research, the cognitive model outlined by Flower and Hayes will serve as the 
theoretical base for this study. Practicality has also influenced the choice: as powerful 
as the art of writing is to devoted expressionists who believe writing cannot be taught, 
many more students continue to fill classrooms needing to acquire the strategies 
inherent in effective written communication. Thus, a useful marriage is suggested 
between the cognitive process of writing and the cognitively-based models of 
instructional design. 

The Chara cteristics of Novice and Expert Writers 

To support this pragmatic goal, a deeper analysis of the literature of 
composition relating to the development of expert strategies in novice writers follows. 
Additional research into the cognitive processes has expanded our understanding of 
the goals and strategies of writers. Seminal work by Bereiter and Scai'damalia (1987) 
identify two main cognitive strategies employed by writers: "knowledge telling" and 
"knowledge transforming." The first can be likened to everyday conversation in 
which a person typically responds to topics and shares content. Consequently, it 
relies on a partner's input for feedback and generation of new topics. Knowledge 
transforming, however, refers to something beyond the simple re-telling of 
knowledge. A rephrasing of Bereiter and Scardamalia's examples analogize the 
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difference between knowledge-telling versus knowledge-transforming cognition: 
everyday thinking versus logical reasoning, recreational reading versus critical 
reading, and spontaneous singing versus a soprano's aria (1987). Thus, although 
writing a diary entry may not demand all the resources an expert has available, more 
knowledge-transforming tasks such as writing a sonnet or a persuasive essay do 
require tlie full range of subskills. The main difference, then, between knowledge- 
tellers and knowledge-transformers is the strategy employed by each. Though both 
attempt to communicate content related to the topic, the knowledge-transforming 
writer uses an additional problem-solving process to integrate higher-level rhetorical 
goals along with the simple delivery of content (Bereiter & Scardamalia, 1987). 
These authors conclude that transforming knowledge requires a differently structured 
model from knowledge-telling. The former's model contains as its distinguishing 
features two embedded problem- solving processes, one involving the rhetorical goals 
of presentation and the other directed at content. 

Further research into the differences between novice and expert writers 
supports the findings reported by Bereiter and Scardamalia. In order to clearly 
distinguish these differences, the following section examines five main areas: 
dominant procedure, goal, method, focus, and writing structure. As suggested above, 
the novice, or knowledge-teller, uses a dominant procedure driven by the goal of 
generating text. The procedure involves a relatively simple method of memory 
probes followed by text generation (Bereiter & Scardamalia, 1987) and is pursued so 
routinely by novice writers that it attains a level of functional automaticity 
(McCutchen, 1988). Thus, writing occurs as a non-conscious act and creates what 
McCutchen refers to as an ^'encapsulated procedure.'' A "telling" example cited by 
the author comes from the writing protocol of a second-grader: *'A11 my sentences 
start with 'my,' so I should just put it there" (p. 316). Also, because an encapsulated 
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procedure requires the least mental effort, novice writers are not motivated to change 
to the more cognitiveiy taxing knowledge-transforming strategies. 

Thus, for knowledge-tellers, the preferred method of text generation is a 
highly routinized procedure of "retrieve-and-write" (McCutchen, 1988) that provokes 
little need to question, set goals, or assemble an argument In fact, McCutchen posits, 
because automaticity may lead to "encapsulated language-generation procedures" (p. 
308), the novice vmter's very strategy bars itself against the metacognitive processes 
that foster more skilled writing. Relatedly, because novices look within to memory 
and respond to internal cues, it is not surprising that their writing becomes what 
Flower has referred to as "writer-based" prose (1981c). That is, "writing that may 
seem quite clear and organized to the writer but is not yet adequately designed for the 
reader" (p. 147). This reminds one of Lord Byron's prescient remark that "easy 
writing makes for difficult reading." Two basic features of writer-based prose are an 
egocentric focus on the writer and a narrative organization that reflects the writer's 
own experience of the topic (Rower, 1981c). Lastly, because of the ingrained think- 
say method (Bereiter & Scardamalia, 1984a), the structure of this writer-based prose 
often resembles a list of ideas related to the topic, rather than an integrated 
composition. To summarize, novice, or knowledge-telling writers tend to use a 
functionally automated, content-driven, retrieve-and-write procedure and 
consequently generate choppy, list-like writing that reflects their self-centered focus. 
An example of this sort of writing is taken from McCutchen (1988): "'Swimming is 
fun, but swimming can be dangerous too. I like to swim under water. My dad is the 
swimmer. Sometimes my brother dunk's (sic) me. My mother make's (sic) me swim 
back and forth over and over'" (p. 314). 

As expected, the characteristics of expert, or knowledge-transforming, writers 
contrast sharply with those of novices. An examination of the same five areas of 
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dominant procedure, goal, method, focus, and writing stmcture reveals interesting 
differences. First, mature writers are not locked into an "encapsulated" procedure as 
are novices, but demonstrate self-regulated executive control that allows them to shift 
between writing processes as the need arises. Flower et al., (1989) identified three 
main executive-level planning strategies that successfully guide goal-setting, local- 
level strategizing, and nesting of subprocesses. In short, three executive strategies 
promote rhetorical goals that influence how sections of a writing contribute to the 
whole; prompt word-choice decisions at the sentence level; and organize when and 
how planning, translating, and revising are implemented. Considering the cognitive 
load inherent in such problem-solving, it is not surprising that expert writers often 
remark on the difficulty of writing. In describing one expert writer studied, 
McCutchen noted protocols showing writing as a "struggle," because the subject's 
language-generation procedures were guided by her "goals to reach her audience and 
to create a coherent, well-written text" (1988, p. 315). 

The fact that expert writers even choose to engage in these more cognitively 
taxing activities comes from the different goal that drives their process. Rather than 
merely wanting to generate text, as is the case with novice writers, the expert is 
motivated to effectively integrate such rhetorical goals as persuasion, effect on 
audience, and voice, with the communication of content. Linda Flower and John 
Hayes (1981a) used thinking aloud protocols to determine that good writers generate 
60% of their new ideas based upon rhetorical goals while poor writers derived 70% of 
their new ideas from either the topic alone or in response to the last idea under 
consideration. 

The expert writer's executive-level control of writing subprocesses and 
rhetorically motivated goals naturally invoke the recursive, interactive, problem- 
solving method associated with composing. Expert writers typically plan and monitor 
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such global goals as purpose or audience while simultaneously making word-choice 
selections and editing previous sentences. All these larger concerns illustrate the 
expert writer's reader-based focus. Unlike the novice's writer-based, egocentric focus 
and narrative organization, reader-based writing tends to focus on major questions 
readers might have, uses a hierarchical organization that makes relationships between 
ideas clear, and cues readers into the point of the writing (Flower, 1981c). Thus, once 
again, the writer's focus clearly influences the structural composition. Whereas 
novice writers wrote for themselves in a think-say format that produced list-like 
compositions, expert writers' compositions reflect a more effective hierarchical and 
reader-cueing organization because they focus on connecting with the reader. 

Yet, beyond these stark contrasts, expert writers engage in activities that 
novice writers do not even attempt. Bereiter and Scardamalia (1987) identified four 
key activities that further distinguish novice knowledge-tellers from more expert 
knowledge-transformers. First, start-up times are considerably greater for more 
sophisticated writers. Whereas novices need only to retrieve ideas, and thus begin 
almost immediately, expert writers start with the planning inherent in their problem- 
solving strategy. Second, expert writers often engage in a form of notetaking that 
often bears little resemblance to what becomes their written draft. Third, thinking 
aloud protocols of expert writers reveal the kind of higher-level, interactive, self- 
regulation described by the knowledge-transforming model, whereas knowledge- 
telling writers had protocols that mirrored what was written. Related research by 
Matsuhashi (1981) found that when writers are involved in the more complex 
processes of persuasion or evaluation, they pause significantly longer than when 
using the knowledge-telling strategies of reporting or narrating. Lastly, expert writers 
tend to revise their compositions because they view them as a natural progi'ession in a 
decision-filled process. Thus, besides strong contrasts in dominant procedure, goal. 
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method, focus, and structure, expert v^Titers also differ from novices by engaging in 
more thinking, notetaking, problem-solving, and revising. Table 1 illustrates these 
differences. 

Helping Novice Writers Develop Expert Characteristics 

Clearly distinctions exist between novice and expert writers. The question 
then follows: how do we move a novice from the knowledge-telling stage to the more 
sophisticated level of knowledge transformation? Deborah McCutchen's work on 

Table 1 

Comparin g the Characteristics of Novice and Expert Writers 



Novice Characteristics Expert Characteristics 



An Encapsulated Procedure 


An Interactive Process 


• Functional automaticity guided 


• Executive control over subprocesses 


by content 


guided by content and rhetorical goals 




(Bereiter & Scardamalia; McCutchen) 


• Goal: Text generation 


• Goal: Rhetorical effectiveness (Bereiter 




& Scardamalia) 


• Method: Fixed and automatic 


• Method: Recursive and interactive 


retrieve- and-write strategy 


problem-solving strategy (Flower, 




Bereiter & Scardamalia) 


• Focus: Writer-based 


• Focus: Reader-based (Rower) 


• Structure: Listing 


• Structure: Global goal guides subgoals 




(Flower, 1989) 


• Limited start-up time 


• Thoughtful start-up time * 


• Limited notetaking 


• Notetaking* 


• Limited higher cognition 


• Higher level cognition* 


• Limited revision 


• Revising* 




* (Bereiter & Scardamalia) 
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functional automaticity highlights the fact that this transformation does not come 
easily (1988). As described above, skillful writing comes out of a sensitivity to 
rhetorical goals and a focus on coherence. For this to occur, the 'language must be 
directed and constrained by the writer's higher-level goals." What makes this 
subjectior to metacognitive strategies so difficult is the fact that 'language-generation 
procedures cannot be automated (or encapsulated) and still remain available to the 
other subprocesses witli which they must interact" (p. 308). The point is further 
reinforced by tlie earlier-cited research of Bonk and Reynolds (1990) who were left to 
explain the lack of a significant benefit from computer-assisted prompting by 
venturing, "perhaps students' content generation coping strategies were not easy 
habits to change" (p. 49). Therefore, an instructional strategy, if it is to be 
successful, must fundamentally alter the writing process from one driven by a 
"retrieve-and-write" strategy to one where executive decisions are made based upon 
such goals as audience, purpose, coherence, and style, 

Bereiter and Scardamalia (1985) conducted parallel work on moving 
knowledge-telling writers toward more knowledge-transforming strategies. They 
suggested that intermediate writers could benefit from what they labeled procedural 
facilitatiofiy "external aids designed to reduce the processing burden involved in 
bringing additional regulatory mechanisms into use" (p, 566). While the intermediate 
writer is beyond relying on a partner for conversational support, the mechanical 
difficulties of text generation can stress cognitive reserves and eliminate chances for 
the self-monitoring and reflection inherent in more mature writers (Bereiter & 
Scardamalia, 1984b), However, provided with the support of a prompting question or 
idea, the writer may bridge to more advanced outcomes. 

In one particular study investigating the ability to teach reflective processes, 
Bereiter and Scardamalia (1984b) used decks of cards that listed planning cues for 



opinion essays. The phrases the experimenters used as procedural facilitators were 
divided into five categories. Table 2 shows these categories and sample facilitators 
from Bereiter and Scardamalia's study (1984b, p. 180). The cards were used by 
sixth-grade students throughout a long-term experiment. Cards were used for both 



Table 2 

Sample Categories and Related Procedural Facilitators 



Category 


Sample Procedural Facilitation 




"An important point I haven't considered yet is. 


NIpvw THpa 


"No one will have thought of 




"A better argument would be. . 




'Tm getting off topic so..." 


Improve 


"I really think this isn't necessary because. . 




'To liven this upTU..." 




"An example of tliis is. . 


Elaborate 


"I could develop this idea by adding. . 




"My own feelings about this are. . 




"A goal I think I could write to. 


Goals 


"My purpose...'' 




"I can tie this together by. . ." 


Putting it Together 


"My main point is..." 




"If I want to start off with my strongest idea I'll. . ." 



Note : From 'Teachability of reflective processes in written composition" by C. Bereiter and M. 
Scardamalia, 1987, Cognitive Science, 8, p. 180. Copyright 19P4 by the authors. Adapted by 
permission. 



public "soloing" demonstrations (where one student brainstormed before the entire 
class) and individual planning sessions. They augmented extensive modeling of 
thought and direct insttiiction of strategies. The combined treatment produced 
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significant increases in reflectiveness as measured on a 9-point scale. Specifically, 
the treatment group averaged 5.43, compared to the control group mean of 3.35. 
Interestingly, a more detailed analysis of pre- and post- tests revealed general 
improvements in all but two of the eleven key descriptors, those relating to 
^'distinctive viewpoint" and "distinctive manner of presentation" (p. 180). 
Additionally, informal observations of the treatment group revealed, among other 
things, increases in students' ability to monitor and analyze thinking, use goals for 
selecting ideas, reflect more on information sources, and perform more mature 
notetaking. This study suggests that procedural facilitators used in combination with 
direct instruction and reinforcement can positively impact student writing. It also 
draws attention to the possibility that extensively modeled, overt strategizing may 
dampen a writer's distinctive voice and style. 

Any analysis of procedural facilitation must include a discussion on the 
purpose of the facilitating prompt. Bereiter and Scardamalia make an important 
point: as one migh: expect, immature writers, focused on "knowledge telling," 
typically respond to prompting by simply generating more think-say text because 
their goal involves producing content, not problem- solving rhetorical or 
presentational concerns (1985). The point at which intermediate writers revert to a 
knowledge-telling strategy has been referred to as "downsliding" (Bruce et al., 1982) 
and indicates that the precise nature of the prompting is significant. Appropriate 
procedural facilitators must redirect the cognitive strategies of the intermediate writer 
toward broader goals of the writing task. Although Hayes and Flower (1981) suggest 
that writers downslide when they reach maximum cognitive load, Bereiter and 
Scardamalia (1987) explained that it is not information processing capacity that 
novice writers lack, but the fall tool kit of strategies that allows the expert writer to 
problem-solve. The goal of effective prompting is thus to encourage the developing 
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writer to stretch toward more knowledge-transforming strategies. Prompts, then, 
should be geared toward modeling a wider range of strategies to help the writer 
become more introspective, aid content production, maintain focus on the topic, assist 
in the creation of executive plans and goals, atter d to rhetorical decisions, identify 
errors in the text, and promote coherence (Scai'damalia and Bereiter, 1985). Besides a 
clarity of purpose for procedural facilitators, their wording or content is also 
important and is discussed later in the chapter. 

This discussion of potential strategies for moving knowledge-telling writers 
toward a knowledge-transforming strategy must conclude with two caveats from the 
researchers most involved with the problem, Bereiter and Scardamalia (1987) first 
caution that short-term interventions can not hope to bring about the restructuring of 
cognitive processes needed to turn a knowledge-teller into a knowledge-transfonner. 
In fact, their previously discussed treatment for teaching reflectiveness was conducted 
in two 45-minute periods a week for 15 weeks. This echoes the previously cited 
comments of others (Bonk and Reynolds, 1990; McCutchen, 1988) attesting to the 
impenetrability of novice writers' non-conscious, retrieve-and-write routines. 

The second warning offered by Bereiter and Scardamalia (1987) also warrants 
notice. Although the problem- solving process of the knowledge-transforming writer 
is preferable as a more resourceful cognitive strategy, the assumption should not be 
made that writings composed by knowledge-transformers are inherently better than 
those written by knowledge-tellers. These strategies merely refer to **mental 
processes by which texts are composed'' (p, 13). It must be remembered that the 
success of a beginning knowledge-transformer's problem-solving is questionable 
because it relies on the novice's limited expertise. Bonk and Reynolds (1990) offer 
one indication, however, that students with a greater awareness of their writing 
strategies do produce higher quality writings. These researchers created the Index of 
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Writing Awareness that assesses middle school students' metacognition about their 
own writing strategies. Details concerning this instrument are presented in the 
method section, but it is important to note that some research does suggest a 
correlation between a students' cognition and the quality of their writing. If increases 
in IWA scores are found to correlate to increases in writing quality, then evidence 
will exist to support the knowledge-transforming strategy as qualitatively beneficial. 
To conclude, even though one's in-process cognitive strategy is no clear indicator of 
effective writing, evidence has determined that students with more savvy about how 
they write tend to produce better texts. 

As a brief footnote to the topic of procedural facilitation, Bereiter and 
Scardamalia (1987) make a distinction between procedural and substantive 
facilitation. The latter can be typified by the example of teacher-written comments, 
suggestions, or hints on a student's paper as well as spelling- and grammar-checkers. 
Thus, substantive facilitation is directed at something the student has already written. 
Because of this focus on a specific topic or problem, substantive facilitators reduce 
the actual cognitive burden for the learner (Bonk & Reynolds, 1990). Bereiter and 
Scardamalia tend to shun substantive facilitation because it often does the thinking for 
the novice, rather than guiding the learner to his or her own cognition. Although this 
perspective is valid, the author of the present study sees the two facilitators as 
working toward different ends. Procedural facilitators guide leamers toward new 
strategies during the process of drafting while substantive facilitation asks students to 
examine specific choices during revision. 

To summarize, cognitive research on the writing process has proposed a 
descriptive model whose key characteristics involve interaction between a variety of 
subprocesses over which an expert writer maintains self-regulated control. Further 
research on tliis cognitive model has described novice writers as those who work from 
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a non-interactive, non-conscious, retrieve-and-write strategy. Instructional practices 
of intervention and facilitation have been suggested to guide these novice writers 
toward using the more advanced strategies inherent in the writing process of expert 
writers. Although promising research has shown statistically significant effects from 
procedural facilitation, the research community has yet to extensively investigate the 
strategy when implemented in computer-guided writing programs. 

A Survev of Computer- Assisted Writing Software Programs 

Computer-assisted writing programs guide learners through the phases of 
planning, drafting, and revising a paper. These programs are promising in that they 
attempt to support the writing process approach to composition (Montague, 1990). 
Typical among the various features of the programs are pre-writing activities such as 
listing, outlining, schematic mapping, goal-setting, and brainstorming, which serve to 
generate ideas and promote broad executive strategies. Prompts during the drafting 
phase may guide writers to problem-solve decisions congruent with those employed 
by more mature writers in the specific genre. Once a draft is written, another whole 
type of software serves to prompt writers to evaluate their product and make 
revisions. Besides a linear approach that sequentially leads a writer through a pre- 
specified process, another possible altemative supports the recursiveness of the 
writing process by allowing a writer at any moment to plan, compose, or revise, and 
therefore, provides various prompts available upon user-demand. 

Over the past 10 years an array of computer-assisted writing programs have 
been developed by researchers around the country. Reviewing a few of these 
programs reveals a general consensus within the writing community regarding key 
features of such programs, and highlights interesting ways guided-writing software 
has been used to promote expert strategies in novice writers. 
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DRAFT 

Developed at Carnegie-Mellon University by Christine Neuwirth (1984), 
DRAFT guides writers during the process of composing, aids teachers in diagnosing 
problems and fostering change in student writings, and allows researchers to record 
the evolving process, DRAFT uses the idea of separate windows, one for the current 
writing, and one for notes or drafts. Otherwise, the software prompts the writer 
through creation c f an essay and then allows the teacher to make annotated comments 
within the document. Thus, aspects of DRAFT that promote expert strategies are its 
goal-directed, in-process prompting that encourages a reader-based and rhetorically 
effective essay. Also of interest is DRAFT'S notetaking feature and its use of 
substantive facilitation via teacher annotations. 

CATCH 

Based upon research by Colette Daiute (1985) at Harvard University, 
CATCH is an interactive program designed to assist students in analyzing their 
writing for correctness and structure, that is to say, exclusively "evaluative" or 
revision processes. It used a combination of procedural facilitators to provoke 
analysis of the text and substantive facilitators to highlight vague or unnecessary 
words. The software's interactive nature was thought to develop a sense of audience 
in the writer. Inherent in the program's revision strategy was a goal to promote more 
highly developed executive control over the subprocess. Daiute predicted that **the 
more revising the subject did, the more self-monitoring he or she had done" (p. 135). 
She found that, **After using CATCH, most subjects made more types of changes in 
their texts and corrected more errors" (p. 136). Results, however, indicated that the 
Ality rating of the computer-assisted texts were no higher than the hand- written 
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post-test A weakness of Daiute's work is the extremely small sample she used (eight 
subjects), the generic nature of the prompts ("Does this paragraph have a clear 
focus?"), and the stilted interface (i.e., *'If yes, press y; if no, press n"). Yet even with 
these limitations, CATCH models or prompts many key characteristics of the expert 
writer. First, its interactive process mimics the recursive nature of mature writing. 
Also, the facilitators direct the writer toward more rhetorically effective writing with 
a focus on both correctness and audience. Lastly, by targeting the subprocess of 
revision, CATCH engages novice writers in an activity typically performed only by 
expert writers. 

WANDAH rcurrentlv available as "HBJ Writer"-! 

Developed by Ruth Von Blum and Mike Cohen at UCLA (1984), 
WANDAH (Writing- Aid AND Author's Helper) is based upon the research of 
writing. Von Blum and Cohen began by analyzing the weaknesses of using word 
processors alone as a compositional-assist. The authors found that composing with 
word processors promotes less-structured, rambling pieces; that revisions are at the 
surface level; and that students carry their "style" or writing strategies to the software, 
thus not significantly altering their knowledge-telling strategies (Von Blum & Cohen, 
1984). WANDAH itself is an integrated writing system that ha^ tfiree major 
components: a word-processor designed expressly for on-line composing; a set of 
computerized pre-writing aids to encourage planning and the free flow of ideas; and a 
set of aids to facilitate reviewing and revising the work thematically, stylistically, and 
grammatically. While many of WANDAH's features are supported by a cognitive 
approach to writing theory, such as Linda Flower's "Nutshelling,'' other modules are 
designed to promote fluency and thus encourage knowledge-telling. Among tliese are 
the modules on "Invisible Writing'' (writing with the computer monitor turned off) 
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and **Freewriting'' (writing quickly without interruption so as to quiet the writer's 
inner editor). While developing fluency holds an important place in a writing 
program, it addresses exclusively generative issues that are outside the scope of the 
current study. The "Planning" program does, however, work towards knowledge- 
transforming ends by guiding the user through the overall structure of an essay (title, 
thesis statement, arguments in support, and counter-arguments). The student then 
arranges the ideas into an outline. In the area of revision, WAND AH uses substantive 
facilitators such as spelling- and grammar-checkers. Besides these, it also employs 
another "nutshelling" aid that asks students to wait at least half a day before summing 
up the argument of their paper into another **nutshell." Students then compare this 
second nutshell to what they have actually written. An "overview summary outline'' 
and "transition word" aids also help the students check the structure of their writing 
and focus on the words they use to move from one topic to another. In summary, 
WAND AH is a more complete computer-assisted writing program than DRAFT or 
CATCH. It promotes many characteristics of mature writers with its interactive, 
problem-solving approach, focus on audience and rhetorical effectiveness, and 
emphasis on structured organization of thought. WANDAH's most distinctive 
features are its focus on planning and revising, two activities with which the novice 
writer is little acquainted. 

Formal research of WAND AH focused predominantly on its cost 
effectiveness as a replacement for teachers. The Utah State Office of Education used 
WANDAH to technologically move English instruction from a grammar-based 
curricula to the process approach as dictated by a new language arts framework. 
Researchers found that students were writing more, that teachers liked reading typed 
papers, and that the writing was as good as that produced with a teacher (Shaver, 
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1986). These findings provide one of the few formal measures of computer-guided 
writing. 

QUILL 

Developed by Andee Rubin and Bertram Bruce (1984), QUELL is a set of 
programs to assist writers from grades 2 through 12, Of particular interest is one 
module called "The Planner" which helps students plan and organize their 
compositions. The Planner essentially serves to outline students' thoughts via 
prompting questions (i.e., "What is the name of the book?" and "Who is (are) the 
main character(s)?"), thus guiding learners interactively through a process targeting 
clarity and broader rhetorical goals. A second module is called "Story Maker" and 
helps students write stories by choosing previously wri-. ^n story parts and putting 
them together. This feature was inspired by Bruce and his theory of "downsliding," 
where a struggling novice writer shifts focus away from idea generation and higher- 
ievel organization to the lowest level of writing (handwriting, punctuation, etc.). The 
sorting and organization of the story parts demands that students think more globally. 
Story Maker's strategy gains further support from Bereiter and Scardamalia (1987) 
who suggested giving students a complex last line to a writing, thus making them 
work with a structural goal rather than the typical formless chronology of the 
knowledge-teller. Therefore, Story Maker forces novices to engage in the kind of 
problem-solving that lay at the heart of an expert's strategy. While Rubin and Bruce 
(1984) present a useful summary of research on computers and writing, their purpose 
is to challenge more research, rather than reveal findings either on others' work or 
their own. 
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Writers^ Helper n 

Perhaps the most successful computer-assisted writing program was 
developed by William Wresch (1984) at the University of Wisconsin Center, 
Marinette. Initially created as '*EssaywTiter'' for the Apple II Plus, the program under 
its current name is available for several computer platforms. Wresch openly shares 
his debt to other researchers, and, with this input, it is not surprising that Writers' 
Helper n is among the most complete computer-assisted writing programs. Unlike 
most of the previously examined software. Writers' Helper n addresses all three 
major phases of th^ writing process, with modules for pre-writing, drafting, and 
revising. Because many of the modules are designed to promote fluency rather than 
transformative writing, only the modules that advance more complex strategies are 
discussed. 

First, in the area of planning or pre-writing, ''Lists'' prompts students to create 
list upon list in an effort to encourage deep thinking about a topic rather than 
accepting the first retrieved idea. This builds upon novice writers' tendencies to think 
in a list-like way, but challenges them to go beyond the first idea they think of so that 
the students may choose tiie best ideas based upon their goals. 'The Questic^ner" 
provides a series of 20 questions about a range of subjects to prompt thinking and 
offers random feedback to promote an interactive, audience-based response. A 
second prewriting level is ''Explore a Topic" and contains three other subprograms: 
"Crazy Contrasts," "Trees," and "Three Ways of Seeing." The first asks students to 
compare their topic to randomly presented topics (such as an English muffin, a 
parking meter, and Richard Nixon) in order to lead them to new insights on their 
topic. "Trees" prompts students to brainstorm and categorize their topics and then 
displays a tree with the topic as trunk and its various related brainstonned branches. 
"Three Ways of Seeing" is an extended questioning device, that, like the other 
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"Explore a Topic'' modules, prompts students to reflect on their topic and shape their 
planning with audience and structure in mind. At the third level of prewriting, a 
student chooses from a variety of organizational structures (Cause / Effect, 
Comparison / Contrast, etc.) and is stepped through identification of subject, 
audience, and purpose, followed by labeling of support and topic sentences. These 
prewriting programs promote the sort of cognition and skills that are characteristic of 
more advanced knowledge-transforming learners. 

Besides these prewriting and in-process drafting assists, Wresch's Writers' 
Helper n also includes a standard word processor and "The Analyzer." The latter is a 
set of revision programs that look at homonyms, outlining, readability level, 
sentences, paragraphs, and usage. Wresch had tended to side with other researchers 
who shy away from using substantive facilitation, saying, "You know the concern. 
What would Writer's Workbench or Grammatik say to Faulkner: Shorten your 
sentences? Use fewer prepositions? Be less parenthetical?" (p. 151). Yet, Wresch 
opted to include these feedback features as a way to give "useful, though limited, 
information to students about their work, [and] to create new opportunities for 
teachers to discuss elements of style tliey normally can't interest students in" (p. 151). 
Thus, Writers' Helper II guides students through all the key activities of an expert 
writer as delineated in Table 1. Although Wresch mentions a then-future 
experimental study of Writers' Helper II, a review of the literature could not find any 
reference to its taking place. One potential drawback to a formal study of Writers' 
Helper II is that because the program is so flexible and contains so many modules, the 
fine distinctions needed to begin establishing theoretical foundations would be 
difficult to make. 
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Wordsworth II 

Designed by Cynthia Selfe and BilUe Wahlstrom (1982) at Michigan 

Technological University, Wordsworth II is the most interactive computer-assisted 

writing program surveyed. The program was created to lessen the workload of 

composition instructors (teaching within the more labor-intensive writing process 

model) by allowing the computer to individualize instruction. Wordswortli II covers 

all three phases of the writing process with **Planning'' and **Polishing" modules 

bookending the drafting phase. As stated, this program is more interactive with 

introductory review and instruction on **lecture points" for the specific writing 

domain (description, narration, personal writing, classification, evaluation, 

persuasion, literary analysis, and creative writing). By engaging the user in an 

instructional dialogue, key strategies or points can be reinforced: 

Well, Horatio [the actual student's name]. That's a good start, but you 
could probably use a few more ideas in your brainstorming list. 
Remember, the whole idea of listing is to generate lots of ideas that you 
can choose from later on. (p. 9) 

Besides this interactivity Wordsworth II's polishing modules offer the learner 
three levels of revision assistance. Division into early, middle, and later drafts allows 
the computer guides to target, and thus promote, more useful strategies for revision. 
The early drafts are checked for such global features as purpose, audience, avid major 
strengths and weaknesses. Middle drafts are assessed on a more specific level, 
targeting organization, effective introductions, and dialogue. The later drafts are 
examined for the details of transitions, verb-use, punctuation, and spelling. A final 
characteristic of note is Wordsworth IPs lively "voice." Prompts are relatively free of 
composition jargon and demonstrate a flair for creativity. However, although the 
program voice seems more user-friendly, the actual user-interface requbes distracting 
keystroke inputs (i.e., "Do you need practice in recognizing the main idea of a 
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paragraph? If you do, type *yes\ if not type 'Go On\" p, 6). In closing, at the time 
of the cited report, no formal investigation of Wordsworth II's effect on writing 
quality had been conducted. The authors did, however, note positive feedback from 
learners in terms of satisfaction after using the program. 

A Formal Studv of Computer-Guided Writing 

The only formal study of computer-assisted writing found in a review of the 
literature, however, dampens the enthusiasm generated by the above array of 
research-based writing programs. Bonk and Reynolds, in an investigation of middle 
school students' development of writing awareness and performance within a 
"Generative/ Evaluative Computerized Prompting Framework," (1990) attempted "to 
develop and test a theory of prompted writing" (p. 2). In brief. Bonk and Reynolds 
created a program that offered on-demand prompts related to idea generation and text 
evaluation. After junior high school students used the program for a series of essays, 
measures of the treatment groups failed to show any statistically significant 
advantage. Comparing the program to the characteristics of the expert writer reveals 
that it is clearly interactive and promotes recursion in order to produce more 
rhetorically effective, reader-based writings. Thus, while most of the characteristics 
of experts are included Bonk and Reynolds' design, their study's failure to produce 
statistically significant findings suggests that perhaps other critical issues are involved 
in designing effective computer-assisted writing software. 

First among these issues could be the user-interface offered by the technology. 
In the Bonk and Reynolds' study, WordPerfect 4.2 software had been set up with 
macro commands to elicit a series of twenty-four individual prompts. If a novice 
writer labors over sentence construction and spelling as suggested by the Uterature, 
asking the same student to select and perfoirn the unnatural macro "alt- shift letter" 
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command during "reflective moments in the composing process'' (p. 19) may i»r i be a 
performance-sensitive request. 

Besides the cognitively taxing aspect of the macro commands' awkward 
interface, a second problem was the array of prompt types and associated 
categorization. Generative prompts were categorized as fluency, flexibility, 
originality, and elaboration (the authors cited the work of Davis, 1986, and Torrance, 
1974). Bonk and Reynolds translated these categories for student com.prehension as 
''more ideas," "types of ideas," "new ideas," and "extenders," respectively. 
Evaluative prompts were categorized as relevancy, logic, assumptions (recognizing 
bias in one's thoughts), and conclusions. The authors translated these evaluative 
prompts as "quality," "clear/logical," "assuming," and "conclusions." Although these 
categories are supported by research, the cognitive price exacted by trying to 
distinguish the differences among them seem to outweigh any potential benefits. 
Even though research recommends making learning strategies overt (Scardamalia, 
1987), intuition suggests that students might do better with fewer than eight 
categories. 

Aside from the interface and categorization problems, a third potential 
weakness in the study was the generic nature of the prompts used: "Try to combine 
your ideas into something really unique;" "Have you used your creativity or 
imagination?" (p. 59). While preserving procedural facilitation's goal of provoking, 
not presenting, solutions, the generic nature of these prompts may be too general to 
penetrate a novice's "encapsulated" procedure (McCutchen, 1988). Because of their 
lack of metacognitive, problem-solving strategies, novice writers cannot be expected 
to turn "maxims into actions" (Flower, et al., 1985). A comparison to Writers' Helper 
II shows the latter employing a personal or conversational tone and provoking 
creativity and imagination with pointed, unexpected prompts (i.e. how is your topic 
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like an English muffin?). Clearly, as seen in the description of the expert writer's 
activities, not every aspect of prompting should be directed toward such expansive, 
creative idea-generation. But it might be ventured intuitively that an appreciation for 
detail and specificity in prompting might be rewarded with the same in student 
writing. 

A fourth potential failing of Bonk and Reynolds' model of prompting is that it 
never compelled students to respond. Although appreciation of the expert writer's 
complex cognitive juggling act recommends not complicating it further with 
computerized interruptions, the retrieve-and-write routine of the novice demands 
intervention. According to Deborah McCutchen, ^'successful writing instruction 
seems to confront immature writing strategies head-on" (1988, p. 320). Thus, even 
though Bonk and Reynolds chose a user-initiated system of on-line help supported by 
Flower and Hayes' research on pregnant pauses (1981a), the fact that students did not 
have to either access the "guides" or implement textual changes because of prompting 
may not have sufficientiy intervened in the subjects' encapsulated knowledge-telling 
procedures. 

Lasfly, the fact that Bonk and Reynolds hypothesized a difference between 
paper and pencil writings and computer-based compositions raises the issue of media 
comparisons. Richard Clark (1987) has made a strong case against such cross media 
studies. In particular, he states that such comparisons are confounded with too many 
variables to control for. He also notes that innovation in the treatment often lifts the 
control. Because computers have become so available, such cross media comparisons 
may miss an important point: use of computers no longer needs justification; they are 
an integral part of many students' learning processes. 

Thus, to conclude this survey of computer-assisted writing programs, a 
composite view of how programs prompt novice writers into more advanced activities 
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is presented. First, many programs (DRAFT, CATCH, WAND AH, QUILL, Writers' 
Helper n, Wordsworth II, Bonk & Reynolds) emulate executive control over the 
subprocesses of writing by providing on-line support that encourages students to 
move between goal setting, idea generation, text production, or revising. Typically, 
these guides are based on concerns for rhetorical effectiveness and clarity, and thus 
share the goal and focus of the expert writer. The very fact that rhetorical goals exist 
engages the learner in problem-solving strategizing focused on ameliorating content 
and presentation. The most sophisticated programs (WAND AH, QUILL, Writers' 
Helper II, Wordsworth II) encompass the complete writing process and therefore 
promote the recursive reflection and rewriting that stand as a hallmark of the 
knowledge-transforming writer. Particularly in programs that guide a learner through 
a longer writing (DRAFT, WAND AH, QUILL, Writers' Helper H, Wordsworth II), 
the user is exposed to subgoals that support more global goals in order to achieve 
coherence. Several of the programs feature pre-writing or planning modules that 
encourage the start-up time that precedes composition by knowledge-transforming 
writers (WAND AH, QUILL, Writers' Helper II, Wordsworth II). One program 
(DRAFT) offers an on-line notetaking option, several promote learner-emphasis on 
revision (CATCH, WANDAH, QUILL, Writers' Helper II, Wordsworth II) and 
employ substantive facilitators (DRAFT, CATCH, WANDAH, Writers' Helper II, 
Wordsworth II). Computer-assistance typically appears in at least one of the 
following three forms: Prompts focused on a specific segment of writing (i.e., 
introduction, supporting examples, etc.), guiding interventions (comments or 
suggestions that support writing as a process), and on-line guides (i.e., optional 
commands that offer *'help" with spelling, idea-generation, or rhetorical goals). In 
conclusion, although much work has been completed by many researchers in 
designing computer-assisted composing software consistent with the literature of 
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writing, hard evidence has yet to prove whether the programs help students develop 
as writers. 



Table 3 

Computer- Guided Writing Programs: A Summary of Features 



Features 


DRAFT 


CATCH 


WAND AH 


QUILL 


Helper II 


W.Worth 


B&R 


Executive control 


V 




V 


V 


V 


V 




Complete process 










V 






Global goals 








V 








Planning time 
















Notetaking 


V 














Prompts revision 








V 




V 




Sub. facilitators 










V 


V 





Defining the Features of an Effective 
Computer-Guided Writing Program 

The following section defines the features of an effective computer-guided 
writing program based upon the literature of composition, previous software 
applications, and the principles of instructional design. Figure 2 illustrates the major 
goal of computer-guided writing: to guide the novice writer into the activities 
characteristic of an expert writer. The instructional purpose of this effort is twofold. 
First, to promote problem-solving cognition and, second, to produce more rhetorically 
effective compositions. 
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Novice 
Characteristics — ► 

Encapsulated Procedure 

• Functional automaticity 
guided by content 

• Goal: Text generation 

• Method: Fixed and automatic 
retrieval / writing 

• Focus: Writer-based 

• Structure: Listing 

• Limited start-up times 

• Limited notetaking 

• Limited higher cognition 

• Limited revision 




Expert 
^ Characteristics 

Interactive Process 

• Executive control 

guided by content + rhetoric 

• Goal: Rhetorical effectiveness 

• Method: Recursive and 
interactive problem-solving 

• Focus: Reader-based 

• Structure: Sub / global goals 

• Thoughtful start-up times 

• Notetaking 

• Higher level cognition 

• Revising 



Figure 2 

The Role of Computer Guides Within the "Novice and Expert" Constructs 



What is Computer-Guided Writing? 

In defining any nev*' field, one expects to encounter many more questior than 
answers. It would be a mistake, however, not to identify and differentiate among 
these questions because doing so helps focus a definition. The most important 
question facing computer-guided writing software is, "Does it work?" Whether this 
means as an educational experience that promotes more sophisticated cognition or as 
a tool for producing more rhetorically effective writing, guiding must fulfill its 
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promise. If, or when, this first question can be answered in the affirmative, a series of 
questions follow conceming computer-guided writing software: 

Who uses it? Knowledge-tellers? Low-ability or high-ability students? 
Middle, secondary, or college level students? Who likes to use it? Who doesn't like 
it even though their writing improves within a guiding environment? 

What does guiding do? What is its purpose? What types of guides work best? 
What does it mean for a guide to "work best?'* What types of guides result in what 
types of changes? 

When in the writing process should prompting be used? Planning? Drafting? 
Revising? When is guiding not effective? When should prompting be available *'on- 
demand" and when should it intervene? 

Why do computer guides work? Why do they sometimes not work? What 
research findings support their use and what does a descriptive model look like? 

How do computer-guided writing programs work? How do they interact with 
pre-existing strategies? How does cognition begin to prefer one strategy over 
another? 

The present study represents an attempt to begin answering some of these 
questions. 

A Definition of Computer-Guided Writing 

In order to begin exploring these questions, a definition of computer-guided 
writing is needed. Based upon the literature of writing and a survey of existing 
programs, the definition would need to incorporate Bereiter and Scardamalia*s (1985) 
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concept of procedural facilitation, take into account McCutchen's (1988) research on 

the functional automaticity of novice writers, emulate the many characteristics of 

expert writers, and finally execute all of this in such a way as to provoke positive 

change in a novice writer because no matter how theoretically interesting failures are, 

expenditures on educational technology demand instructional returns. Thus, a 

computer -guided writing program is: 

the systematic application of computerized prompts ^ guides^ and 
on-line support designed to effectively move a novice writer toward 
the strategies and/or behaviors of an expert. 

Thus, although designers of computer-guided writing programs may create a 
variety of forms, they do so with an instructional goal in mind. Therefore, computer 
support may prompt, guide, or provide on-line assistance; it may be procedural or 
substantive; it may be offered as an option or required as an intervention; and it may 
be focused on increasing cognition or wilting skills depending upon pre-specified 
educational goals. To borrow the metaphor of Gross (1992), computer- guided writing 
software can be viewed as ''training wheels." Like the aid to beginning cyclists, 
computer-guided writing software gets novices "up and wobbling." The 
technological support can't promise expertise overnight, but through its use, 
beginners go through the same motions as the initiated, thereby starting to develop 
more advanced balancing acts. Similarly, the aids should be abandoned when they 
serve to hamper, rather than enhance, expertise. 

Using an Instructional Design Model 

Because this definition for computer-guided prompting is predicated upon 
effectiveness, it is only natural to turn to the field of instructional design, or 
educational technology, to begin identifying the characteristics of such a learning 
system. As mentioned in the introduction, the field of instructional design creates 
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educational experiences to address pre-specified needs* Although a variety of design 
models exist based upon different learning theories - whether behavioral, cognitive, 
or constructivist — all seek to systemadcally produce a positive change in learners. 

Typical instructional design begins with a front end analysis, or needs 
assessment, to determine whether a need for training truly exists. Certainly, the gap 
between a novice's actual ability and an expert writer's optimal performance warrant 
initiating a training solution (Kaufman, 1979). While a designer would not jump 
immediately to computers as the training solution, with over 2 million, rather idle, 
computers in our schools, they may serve as one effective solution. Purpose-based 
training needs assessment would then follow with an analysis of "optimals, actuals, 
feelings, causes, and solutions" (Rossett, 1987). 

Writing is recognized as a complex, multifaceted task and the composing 
process, from a cognitive perspective, as an individualized problem-solving endeavor. 
Thus, although a thorough analysis of it can not promise a recipe for success, it may 
provide valuable insights that can be applied to writing instniction. Therefore, 
because descriptors exist for both expert (optimals) and novice writers (actuals) 
(Table 1), a next step is to consider the feelings of the leamers. In Computers, 
Cognition, and Writing Instruction (1990), Marjorie Montague reported findings from 
Daiute that when they used computers, "children vvTOte more and stayed with tiie 
writing task for longer periods of time" (p. 71). She goes on to say, "Positive 
attitudes toward computer-assisted composing also were reported by Cheever (1987), 
Boone (1986) and Fernandez (1988)" (p.71). Later Montague relates that, "Other 
studies of secondary level students have noted an increase in motivation; greater peer 
involvement; and more positive attitudes toward instruction, writing ability, and 
revision" (p. 90). Obviously, research indicates that potential users might like 
working with computers. 
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The next area to investigate in a needs assessment is the possible causes for 
the performance discrepancy. The four main types of causes are skill or knowledge 
deficiencies, poor incentives, environmental factors, and lack of motivation (Rossett, 
1987). Certainly, based upon the literature of writing, novices do not have the 
advanced cognitive or linguistic skills required by an expert strategy (Bereiter & 
Scardamalia, 1987, McCutchen, 1988). Relatedly, because the automated, retiieve- 
and-write routine they employ is the easiest (McCutchen, 1988), there is little 
incentive to adopt the more difficult knowledge-transforming strategy. Thirdly, as 
conjectured regarding the Bonk and Reynolds program (1990), sometimes 
environmental factors related to design can frustrate effective use of a computer- 
assisted prompting program. Lastly, although previously cited results showed that 
using computers motivated students in a variety of ways (Montague, p. 90), the 
present author's personal experience suggests that the relationship between 
motivation and computer-use is a more individual matter. The literature of 
motivation theory (Keller, 1979; McClelland, et al., 1953) indicates that not only 
must students see the value of attempting more challenging writing, they must also 
feel confident that they can succeed. With this in mind, any computer-assisted 
writing solution must be at least non-threatening, if not user-friendly. 

The last phase of the needs assessment, **solutions," traditionally involves 
consultation with subject matter experts. The preceding literature review already 
established procedural facilitation and other forms of prompting as promising 
solutions to the current need. Similarly, prevalence of the software applications just 
surveyed also suggests that computer-assisted prompting is an appropriate solution. 
However, unlike traditional training solutions that addressed such well-defined skill 
deficiencies as the steps an expert takes in machining a precision instrument, the 
cognitive perspective views writing as a complex, ill-defined, problem-solving task 
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that takes place in the covert mysteries of human cognition. Thus, to further bolster 
the case, and to achieve a more detailed description of the solution, consultation with 
additional subject matter experts was conducted in the literature of learning theory 
and software design. 

Whereas a behaviorist model of leaming worked well for developing 
procedural training or rote recall, the more complex cognitive task of writing requires 
a more subtle theoretical instrument. First, cognitivism offers the concept of 
schemata, organized networks of prior knowledge (Norman, 1982). Of interest to the 
current study is "top-down" mental processing in which an "empty" schema is 
created, leading a learner to search for appropriate information to fill the cognitive 
vacuum (Hannifin and Rieber, 1989). This schema theory also relates to cognitive 
psychology's notion of "chunking," where people group related instructional 
functions. Thus, if novice writers must begin to see definition in the ill-defined task 
of writing, perhaps a computer-assisted writing program could provide not only the 
larger structural schema (topic sentences, supporting details, etc.), but also examples 
with which to instantiate them. One should recall McCutchen's imperative to 
confront the novice's simplistic and encapsulated strategy head-on. By prompting the 
student to fill an established schema, just this type of positive intervention may occur. 
Similarly, Marlene Scardamalia stated as her first recommendations for designers of 
"Computer Supported Intentional Leaming Environments" (CSILE) (Scardamalia, 
1987) to make knowledge-construction activities overt and maintain a focus on 
cognitive goals. Thus, cognitive models of leaming recommend guiding the student 
toward a more highly organized and developed representation of knowledge. 

A more recently proposed model known as constructivism also offers 
important insights on leaming in ill-structured domains. Rather than see leaming as a 
package of knowing that enters the learner's cognition, constructivism recommends 
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that to gain new knowledge and understanding within a complex domain, learners 
must reconstruct that knowledge within their own cognitive structures (Ausubel, 
Novak, and Hanesian, 1978). Handing knowledge to learners undermines students' 
understanding when they are confronted with the ill-structured domains inherent in 
advanced knowledge acquisition (Spiro, Coulson, Feltovich, and Anderson, 1988). 
Spiro and his colleagues devised ''Cognitive Flexibility Theory" which avoids 
oversimplification and provides multiple representations. Specifically, they employ a 
rich hypertext computer environment that may be searched or manipulated by 
learners, thereby providing resources with which to make meaning. Thus, these 
constructivist views imply that an effective computer-assisted writing program should 
provide a rich environment, suggesting perhaps a bank of examples or choices from 
which learners could construct their own meaning. 

The last area from which to gain insight is software design. In the article 
''Learning with QUILL," (1985), Rubin and Bruce present several lessons for 
software designers. First, "A teacher's instructional philosophy is a more powerful 
determinant of software use than the software itself; that is, the teacher's contribution 
overwhelms that of the software" (p. 20). The importance of the total instructional 
context is supported by Bereiter and Scardamalia (1987) who found that to achieve 
significant effects with procedural facilitation they had to use extensive modeling and 
direct instruction of writing strategies. Second, Rubin and Bruce conclude that, "we 
will all learn more if the software is flexible and adaptable" (p. 26). The fact that 
teachers may want to alter the contents or structure of a guiding program to suit their 
instructional goals suggests that an effective program needs to be adaptable to a 
variety of strategies. Relatedly, because newer, easy to use, authoring systems such 
as HyperCard™ and LinkWay™ are available, classroom teachers themselves may 
serve as designers of computer-guided writing. 
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Another aspect of software design to consider comes from the business world. 
Motivated by a changing world market, companies have begun to flatten 
organizational hierarchies, emphasize performance increases, and empower 
employees with maximum skills in order to be competitive (Rossett, 1991). These 
changes all impact how employees are trained. When high quality training must be 
efficiently replicated, and positive change in performance is demanded, on-line 
electronic performance support systems (EPSS) are preferred over traditional training 
methods (Rosenberg, 1992). Corporate trainers recognize the efficiency of "just-in- 
time" one-on-one training as opposed to the one-to-many information spray typical of 
most classrooms. That the business community views on-line electronic support as 
crucial to survival suggests that perhaps schools and students can benefit from similar 
training experiences. Tlius, even though the power and intelligence of electronic 
performance support systems are beyond the scope of this study, the capital being 
invested in such systems suggests that at its best, on-line, just-in-time, computerized 
learning envh-onments increase performance. 

To summarize, following the model of instructional design, a front-end 
analysis of novice writers reveals important aspects that should inform an effective 
computer-guided writing program. Any instiaictional solution to the performance gap 
between novice and expert writers must balance an apparent affinity students have for 
using computers with theh* deficiencies in skill, lack of incentive to change, and 
uncertain motivation. Also, cognitive and constructivist models of learning 
respectively recommend emulating more advanced cognitive structures and providing 
a rich learning environment. Lastly, previous examples of software designs should be 
considered to create an effective program. 
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Toward a Lexicon of Computer-Guided Writing Software 

Based upon a synthesis of the preceding literature review, survey of software, 
and front-end analysis, some of the questions posed at the beginning of this section 
can be addressed. First, in order to test the effectiveness of this multifaceted 
instructional strategy called computer-guided writing, distinctions between types of 
guidance must be made. Recalling the definition offered previously provides a basis 
for these distinctions: ''the systematic application of computerized prompts, guides, 
and on-line support designed to effectively move a novice writer toward the advanced 
strategies and/or behaviors of an expert." The present study proposes a beginning 
lexicon of three major types of prompts: ''structural chunking," "rhetorical 
interventions," and "on-line guides." 

Structural chunking . The first type of computer guide is borrowed from the 
field of cognitive psychology and refers to the imposition of an expert's structural 
organization on an ill-defined writing task. "Organization of information is a critical 
device for knowledge representation and storage. Experts utilize broader 'chunks' as 
they encode information , . , further [they] can identify more subcomponents to the 
broader group" (Cocking & Mestre, 1989, p. 5). Such foundation in schema theory 
finds empirical support in the existing computer-guided writing programs that lead 
the user through the parts of a particular type of writing. An example of structural 
chunking would be categorizing the introductory paragraph of an essay into 
"attention-getter " "background information," "transition " and "thesis statement" (s^e 
Table 4), Intuitively, any writer who encounters a new type of writing, whether it is a 
reflective essay or a short story, attempts to ascertain the particular movements 
comprising the genre in order to devise subgoals and strategies for achieving the 
larger goal of rhetorical effectiveness. Computer-guided structural chunking must. 
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therefore, do two specific things. First, it must break a writing task into intelligible 
components. Second, because the novice may not have a schema established for this 
chunk, the software must prompt the writer into the type of thinking an expert would 
perform faced with the same task. Thus, structural chunking would be used to initiate 
more advanced cognition as well as improve the rhetorical effectiveness of a novice's 
writing. This last possibility for more effective writing is helped by the fact that the 
final ''chunk" writers encounter is a compilation of all their previous stages assembled 
together. This way, writers work at both sub- and global-level goals for harmonizing 
rhetorical concerns and content. Figure 3 shows the predicted effects of structural 
chunking as they relate to the promotion of expert characteristics in novice writers. 

Rhetorical interventions . The second general type of computer support 
directly attempts to alter the student's cognition while in the process of writing. 
Because the novice strategy of retrieve-and-write is so resistant to change (Bereiter & 
Scardamalia, 1987; Bonk & Reynolds, 1990; McCutchen, 1988), one type of 
prompting must attempt to intervene in this "think-say" routine (Bereiter & 
Scardamalia, 1984). Deborah McCutchen states that the instructional approach to 
changing this novice routine must be, "To intervene in the child's ongoing writing 
process, supplying a new procedural piece that is similar to the more expert 
strategies" (p. 321). Conceptually, the main difference between the knowledge- 
transformer and the knowledge-teller is that the latter only addresses content, while 
the more expert model also contains a problem-solving space for rhetorical goals. 
Thus, a device must be implemented that interrupts the novice routine and redirects it 
toward traits more characteristic of expert writers. Examples of this would be 
unannounced questions that pop-up on the computer screen and ask the user to 
address such concerns as audience, coherence, and voice before allowing the writer to 
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return to the drafting mode (see Table 4 for examples). If combined with structural 
chunking, the interventions could focus on aspects of the particular phase of writing; 
then the student's responses could be compiled in the final ''revision'' screen. In this 
way, students are forced to adopt more sophisticated cognitive activity. Thus, when 
designers have targeted a confirmed group of knowledge-telling writers, they would 
employ rhetorical interventions. Of course, the content of the intervention is variable 
depending on the instructional goal, but because they interrupt the normal train of 
thought and fluency, interventions should promote more knowledge-transforming 
cognition. However, because of the targeted students' rudimentary experience with 
the problem-solving approach to writing, one would not anticipate immediate 
improvements in rhetorical effectiveness through the use of interventions although an 
increase in cognition would be expected. Figure 3 further shows the predicted effects 
of rhetorical interventions as they relate to developing expert charactenstics in novice 
writers. 

On-line guides . The third major type of computerized guidance is inspired by 
the "just-in-time" training found in the new electronic performance support systems 
and on-line ''helps" increasingly featured in most commercial software. This support 
is optional and available to the user upon demand. Within this major type of 
computerized guidance, three particular categories are proposed: "ideas," "strategies," 
and "examples." The first two derive from the work of Flower and Hayes (1981b) 
who stated that the subprocesses of "generate" and "evaluate" can positively interrupt 
an expert's writing process at any time. Thus, to emulate the exper* model, a 
generative pool of ideas as well as in-process evaluative strategies should be available 
to the novice writer as well. Additionally, on-line guides favor the cognitivists' 
schema theory by providing learners with concrete instances to fill a newly created 
schema. This process is particularly applicable when on-line guides are offered 
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within a structurally chunked passage. Table 4 provides examples of generative 
''idea'' guides such as, 'The place looked like../' or "One thing everyone should 
know about this subject is. , Even though content generation supports the immature 
knowledge-telling strategy of novices (Bereiter & Scardamalia, 1987), its inclusion is 
merited on two counts. First, both knowledge-tellers and knowledge-transformers 
probe content sources, so the process is not exclusively a lower level pursuit. 
Secondly, computerized guiding holds the possibility of provoking more expert, 
detailed content, rather than the generalities that can fill a novice's paper. Similarly, 
having a "possibility pool" of ideas just a button-click away encourages leamers' 
creativity (Dodge, 1991) and thus can promote a lively writing process. 

Building on the work of Bereiter and Scardamalia, the second type of on-line 
guide, "evaluative strategies," addresses the concern that procedural facilitators 
should guide knowledge-tellers toward more transformational thinking. Therefore, 
this type of on-line guide offers novices an array of pertinent questions experts might 
ask themselves. Examples offered in Table 4 are: "How does this go along with my 
first idea?" or "What do I want the reader to feel during this part?" Thus, "strategies" 
model the broader executive goals of the expert and potentially prompt novices to 
examine and revise their work in an ongoing and recursive manner. "Strategies" can 
point students to analyze theu* text, verbalize their goals, and problem-solve wiiting 
tasks depending upon the rhetorical constraints of the task (Bereiter & Scardamalia, 
1987). 

The third type of on-line guide, "examples," finds its source in constructivist 
models of learning in ill-defined domains. In order for students to develop their own 
understanding of a complex writing task, a supply of examples should be available 
on-line, thereby providing a rich envkonment of whole "constructs" (see Table 4). 
Like the instantiation of new schema offered by the cognitivist "idea pools" refeiTed 
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to above, the constructivist-inspired guides also offer examples for learners. The 
difference is that some students may learn better by analyzing complete examples of a 
concept (introductory paragraph, "showing writing/' etc.), thereby constructing their 
own schema with its particular characteristics. 

To summarize, "on-line guides" are categorized as aids to generating, 
evaluating, and constructing, or put more simply they are, "ideas," "strategies," and 
"examples." Thus, self-motivated learners, whether pursuing knowledge-telling tips 
on content or knowledge-transforming strategies, have the opportunity to develop 
advanced cognitive abilities as well as creating more expert texts. Figure 3 shows the 
predicted effects of on-line guides as they relate to prompting expert strategies in 
novice writers. In closing, three critical aspects of how these prompts are effected 
must be considered. First, because this type of support is only accessed at the 
learner's volition, the experience of using on-line supports should be facile, and the 
content useful. As was perhaps the case in the Bonk and Reynolds study (1990), an 
awkward or overwhelming interface can stand in the way of a guide's potential use. 
Second, as touched on previously, the content or wording of all prompts is important 
This matter will receive further development shortly. Third, although it is beyond the 
scope of a beginning study such as this to prescribe an optimum number of examples 
or ideas for each type of on-line support, some number must be suggested. Based 
upon the general agreement that people are able to hold an average of from five to 
seven items in short-term memory, these ranges will serve as a guide, witli the actual 
number depending upon the typical length of a category's prompt (i.e., less 
"examples" and more short "idea" hints). 

The content of computer guides . The effectiveness of computerized guides, 
as previously discussed in the analysis of the Bonk and Reynolds study (1990), relies 
on the specific quality of computer guiding. Unlike in-person conferences with a 
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teacher, computer guides have only one chance to make their point. Thus, not only is 
the wording important in terms of vocabulary and voice, but also, because prompting 
serves to model advanced cognition, it must demonstrate specificity. Though many 
other variables need to be considered in making a comparison between studies, a 
simple analysis of procedural facilitators in the Bonk and Reynolds experiment 
(1990) and the study by Bereiter and Scardamalia (1984b) is informative. 
Representative of the former study's prompts are, ''Read the first and last sentences to 
each paragraph. Are there transitions from one sentence to the next?" (p. 60) and 
'Try combining two or more of your ideas into something really unique. Have you 
used your creativity or imagination?" (p. 59). Corresponding prompts in the Bereiter 
and Scardamalia study are, "I can tie this together by. . and "I sometimes wonder. . ." 
(p. 180). Three clear differences exist in terms of length, voice, and person. The 
former prompts are long, speak in general maxims, and use the second person. 
Bereiter and Scardamalia's (1984b) statistically effective latter prompts are short, 
reflect a user-like voice, and speak in the first person. If student response to prompts 
is optional, a strong case could then be made that the wording and content of the 
prompts must be compelling and speak directly to the users. 

Summary of guide types . Three major types of computer guides are 
suggested: structural chunking, rhetorical interventions, and on-Une guides. Three 
subcategories of the latter are ideas, strategies, and examples. Table 4 identifies 
thedominant effects of the types of computer-guides suggested by this study. Figure 
3 shows that key characteristics of the expert writer have been targeted by specific 
computer guides. 
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Table 4 

Types. Examples, and P redicted Effects of Computer Guides 
(Main Effects are in Boldface) 



Types Examples Predicted Effects 

An Intro paragraph chunked as: • Advanced cognition 
Structural • "attention-getter'* 

. • "background iniV • Increased rhetorical 

Chunkmg • "thesis statement" effectiveness 



• "What do you want readers to be • Advanced cognition 

Rhetorical feeling now?" 

• Potential increases in 

Interventions • "Why are you telling the story in rhetorical effectiveness 

this order?" 



"Examples": a thesis 

• "Everyone must take part in 

recycling if it's to work." • Advanced cognition 

On-Line Guides '"^f ^ ^ „ • increased rhetorical 

• The place looked like. . . effectiveness 

"Strategies" 

• "How did I describe the sighis and 

sounds of the setting?" 



• Advanced cognition 

Full Treatment Combines all three of the above types 

of computerized guiding. • Increased rhetorical 

effectiveness 
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Novice 
Characteristics 

Encapsulated Procedure 

♦Functional automaticity 
guided by content 



• Goal: Text generation 



► Method: Fixed and automatic 
retrieval / writing 



• Focus: Writer-based 



Structure: Listing 



Limited start-up times 



Limited notetaking 



Limited higher cognition 



Limited revision 



Expert 
Characteristics 

Interactive Process 

' Executive control 
guided by content + rhetoric 

' Goal: Rhetorical effectiveness 

• Method: Recursive and 
interactive problem-solving 

► Focus: Reader-based 

► Structure: Sub / global goals 

• Thoughtful start-up times 

• Notetaking 

• Higher level cognition 

• Revising 



Code for Dominant Type of Guiding Used to Effect Changes 

Structural Chunking Rhetorical Interventions On-Line Guides 



1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 



Figure 3 

Types of Computer Guides and Targeted Expert Characteristics 
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Describing Versions of the Guided- Writing Software 

Although each guide-type holds the potential to raise cognitive sophistication 
and increase rhetorical effectiveness, not until the three types are tested both singly 
and in combination can their effectiveness be measured. The reason for this can be 
found in the complexity of the cognitive task and the variety of strategies expert 
writers bring to the act of composing. In testing the guide types, programs employing 
different combinations of computer guides were used. One treatment employed only 
structural chunking, another added rhetorical interventions to the chunking, and a 
third combined on-line guides with structural chunking. The fourth version, called 
the full treatment, combined all three of these guides, A fifth variation served as the 
control and combined prompts from the different screens of the chunked program into 
a scrolling text field resident on one screen. Table 5 lists these programs along with 
their abbreviations. 

Table 5 

Versions of the Treatment and Their Abbreviations 



Treatment 


Abbreviation 


Chunking only 


C 


Chunking with interventions 


CI 


Chunking with on-line guides 


CG 


Full treatment 


CGI 


Chunked prompts on one screen 


Control 
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Design Principles for Suggested Guided-Writing Software 

Although each of the three suggested guide-types attempts to effect specific 
changes, some principles apply to the general computer-guided writing treatment used 
in this study. First, to reflect the cognitive complexity that computer-guided writing 
software attempts to instill, programs must be interactive, allowing the user freedom 
to employ executive control over the whole range of planning, drafting, and revising 
subprocesses. Also, rather than linearly lock-step the writer through a composition, 
all manner of recursive routes must be available to the user. Relatedly, because the 
majority of planning occurs in-process, no separate pre-writing feature is included, 
although ample start-up and composing times should be allowed. As for the structure 
of the program, chunked sections of the writing may be supported by related 
interventions and/or on-line guides. If used, interventions will engage learners in 
periodic problem-solving and goal-setting, while cognitive dissonance created by the 
intervention may prompt students to access the on-line guides of ideas, strategies, and 
examples. Lastly, the information contained in the chunked prompts and guides is 
directed at both the content goals and the broader goals of reaching an audience with 
an effective rhetorical structure. 

The Context of Comp uter Guided Writing 

How computer-guided writing software fits into the instructional context 
should be addressed. One of the recommendations Bonk and Reynolds (1990) made 
concerning changes to future studies was to integrate computer prompting with 
classroom instruction, echoing Rubin and Bruce's (1985) comment that a classroom 
teacher has a more profound impact on student learning than a piece of software does. 
Even considering both of these recommendations, the present study hypothesizes a 
positive advantage from using the full treatment model of computer-guided writing 
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software in a context of minimal instructional support. Obviously, more leaming 
might occur in a classroom full of metacognitive discussions and analyses of the 
writing process, but the purpose of this study is to begin to define the elements of 
computer-guided writing softv/are, and to do so requires a limited focus. 

Summary 

This description of computer-guided writing began with a series of questions 
in an attempt to outline the field. Next, a definition was offered and three major types 
of computer guides were identified. A front-end analysis then provided additional 
insights for creating a computer solution to the performance discrepancy between 
novice and expert writers. These insights, as well as a review of the literature and a 
survey of existing computer-assisted writing software, set the foundation for creating 
a potentially effective computer- guided writing program. 

Hvpotheses 

Seven hypotheses have been developed from the research questions. 
Primarily, the effectiveness of guided-writing software is tested. Next, four 
hypotheses are suggested to measure the differences among the three types of guides. 
Last, the relationships between the treatment and learner ability and user-satisfaction 
are studied. Each of the hypotheses refer to specific instruments that will be 
described and validated in the method section of this study. 

Research Question 1 : 

Does computer-guided writing software help students produce more 
rhetorically effective compositions? 
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H(i) - Learners in the group using the full treatment (CGI) of 
computer guidance will write essays that rate higher on the 
California Assessment Program's Scoring Guide for Rhetorical 
Effectiveness than learners in the control group. 

No research has yet found that students who use computer-guided writing 
programs show statistically significant gains. This hypothesis, therefore, is based on 
a logical needs assessment (Rossett, 1987) of the characteristics of novice and expert 
writers (Bereiter & Scaidamalia, 1984a, 1984b, 1987; Flower et aL, 1981; 
McCutchen, 1988) and the subsequent generation of a model for effective computer- 
guided writing programs. The California Assessment Program Scoring Guide for 
Rhetorical Effectiveness was used to measure the hypothesized change. 

Research Question 2 : 

Are there definable types of computer guides that promote specific expert-like 
performances in novices? 

H(2) - Learners in the treatment group with only structural 
chunking (C) will produce writings that rate higher on the 
California Assessment Program's Scoring Guide for Rhetorical 
Effectiveness than learners in the control group. 

Research has indicated that tlie characteristics of novice writers contrast 
sharply with those of experts (Bereiter & Scardamalia, 1984a, 1984b, 1987; Flower & 
Hayes 1981a, 1981b, 1981c; McCutchen, 1988). Thus, an analysis of these 
differences suggests several types of prompts or guides that could promote expert 
traits in novices. Applying different combinations of computer guidance will allow 
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distinct effects of the various guides to be measured because if a model is to be 
proposed, its parts must also be described and their functions illuminated. Hypothesis 
2, then, is a further refinement of the first It differs, however, in its attempt to 
distinguish the rhetorical effectiveness of structural chunking. Because chunking 
forces novices to write to the semantic units that an expert might formulate, the 
rationale here is that novice writers will at least address the critical attributes of a 
particular genre of writing. Although the potential output might leave something to 
be desired in terms of transitions and coherence because this is the most basic level of 
computer guidance, if chunking provides a significant advantage then it can be 
prescribed for effective use in ill-structured writing domains. Support for the 
hypothesis comes from cognitivist schema theory (Cocking & Mestre, 1989; Hannifin 
& Rieber, 1988) and Information Mapping™ (Horn, 1976), 

H(3) - Learners in the treatment group with on-line guides and 
structural chunking (CG) will produce writings that rate higher 
on the California Assessment Program's Scoring Guide for 
Rhetorical Effectiveness than leamers in the treatment group 
with only vStructural chunking (C), 

The focus of this hypothesis is to establish the efficacy of on-line guides. 
Because they model rhetorical goal-setting, and provide examples and ideas, it is 
suggested that the availability of on-line guides will produce more effective writing, 

H(4) -Leamers in the treatment group with structural chunking 
and interventions (CI) will achieve higher scores on the Index 
of Writing Awareness than leamers in the treatment group 
using only structural chunking (C), 
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The rationale for this hypothesis is based on the research of novice writers' 
knowledge-telling, retrieve-and-write, think-say routines (Bereiter & Scardamalia, 
1984a, 1984b, 1987; McCutchen, 1988), Because rhetorical interventions interrupt 
the novice's encapsulated procedure and redirect attention toward more mature 
problem-solving, it follows logically that learners in the treatment group using 
interventions will spend more time engaged in higher level cognition about the 
choices they make in their writing. Therefore, students will achieve higher scores on 
a measure of metacognition concerning their writing process, 

H(5) - Learners in the group with the full treatment of 
computerized guiding (CGI) will demonstrate more expert 
characteristics than learners in all other treatment groups as 
measured by higher scores in each of the following three 
assessments: California Assessment Program's rhetorical 
ratings. Index of Writing Awareness measures, and the degree 
of revision as determined by the Taxonomy of Revision, 

Unlike the previous three hypotheses, this one seeks to measure the 
synergistic effect of the dynamic interaction between the three types of computer 
guidance that comprise the full treatment model. These measures were chosen 
because characteristics such as rhetorical effectiveness, metacognition, and an 
interactive, recursive writing process stand as some of the key strategies of expert 
writers . Although no hard evidence exists, the logical suggestion is that novice 
writers who are developing new cognitive schemas (chunking) and are forced to 
articulate their goals and problem-solving strategies (interventions) will look to 
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available resources (on-line guides) to discover solutions and thus make new 
meaning. 
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Research Question 3 : 

What is the interaction between learner ability and computer support? 

H(6) — Lower ability students in the treatment group with the 
full treatment of computerized guiding (CGI) will demonstrate 
more expert characteristics than low ability students in the 
control group as measured by higher scores in both California 
Assessment Program's rhetorical ratings and Index of Writing 
Awareness measures. 

This hypothesis tests whether low ability students, because they most closely 
match the characteristics of the novice writer, will benefit from the more structured, 
invasive, and supportive writing environment offered in the full treatment model. 

Research Question 4 : 

Is there a relationship between computer support and a student's level of 
satisfaction in using computers as a writing tool? 

H(7) - Students in the full treatment group (CGI) will express 
greater levels of satisfaction in using computers than those in 
the control group. 

This hypothesis tests whether the full treatment addresses the users' 
anticipated needs. Although more mature writing processes are inherently more 
taxing than a novice's automatic procedure, if the model is designed in congruence 
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with a needs assessment, it should address many anticipated difficulties. A Likert 
scale questionnaire will measure students' overall responses to using computers. 

Operational Expression of the Problem 

The present study proposed versions of computer-guided writing software. 
The experiment used to test them employed four versions of the software which 
varied the combinations of guide-types. The treatments and a control group were 
randomly assigned to intact middle school classes who used computers to write 
reflective essays. The rhetorical effectiveness of the writings generated from each 
treatment were measured, as were assessments of writing awareness, computer-use 
satisfaction, and degree of revision. In addition, comparisons were made among 
groups to determine the effects of the three types of guides, thereby refining their 
prescriptive potential. 
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CHAPTER in 
METHOD 



The following chapter describes the subjects, the materials, instruments for 
measuring dependent variables, the procedures, and the experimental design. Last, it 
presents data analysis for each hypothesis. 

Description of the Subjects 

The subjects were one entire "family" (n = 143) of seventh and eighth-grade 
students at O'Farrell Community School: Center for Advanced Academic Studies. 
This family included 68 males and 75 females. Forty are 12-year-olds, 73 are 13- 
year-olds, and 30 are 14-year-olds. O'Farrell is a magnet school emphasizing a core 
interdisciplinary curriculum with high academic standards for all students. It is 
located in an urban section of San Diego and serves a racially balanced, multi-ethnic 
population of lower socio-economic status. O'Farrell was selected for the study 
because students there regularly use an array of new software progi^ams so the threat 
to validity due to reactivity (the Hawthorne effect) would be limited. 

Materials 

Five computer software programs were designed for this study using the 
HyperCard™ authoring system. This section briefly describes each of the programs 
and illustrates them in Figures 4 - 8. The first version of guidance employed only 
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structural chunking (C). This program divided the writing into six major movements 
C*The Opening Occasion,'' "Finding the Absti-action," "Another Personal Reflection/' 
"Exploring the Opposite," "The General Reflection," and "Your Final Awareness"). 
Each section contained a prompt that gave students guidance on what to write. The 
second version added rhetorical interventions to the chunking program (CI), In this 
treatment, questions popped-up randomly 20 to 80 seconds into each chunk, asking 
students to respond with their rhetorical goals for tliat portion of the essay. The 
student's response was recorded and inserted into the revised draft as goal statements 
for each section. A third version added on-line guides to the basic chunking program 
(CG). This program featured three buttons ("Examples, "Ideas," and "Strategies") 
offering students on-demand assistance related to the particular chunk of the essay 
they are writing. The fourth version combined all three types of guides into what is 
referred to as the full treatment (CGI). A fifth program was for the control (Control ). 
To limit cross-media confounding variables, this version of the software used the 
identical prompts as the "chunking only" version, but presented them in one scrolling 
text field on one screen. Figures 4-8 show the five programs. 

Instruments 

Assessing Rhetorical Effectiveness 

Because of writing's multi-faceted nature and students' appropriately 
individualistic responses to the task, attempts to measure student writing challenge 
researchers to choose both the tasks and the assessments carefully. Several factors 
influenced the selection of reflective writing as the domain and the California 
Assessment Plan's Scoring Guide for Rhetorical Effectiveness as the measure (see 
Appendix A for a copy of the scoring guide). First, years of experience working with 
students and colleagues have proved that the higher-order, metacognitive skills 
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Structural Chi*nking 



lUlPrompt: Choose a time your family was going through an importam change 
(one that affected you emotionally). Then use "showing writing" to 
make the reader feel like he or she is there watching what's going on 
with your family. This should be a good-sized paragraph. 
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Screen Design for Structural Chunking (Q 



ERIC 



61 













Ijlfliil 




fjHHjHl 


' Chunking with - Intervention 






Prompt: Choose a time your family was going through an important change 
(one that affected you emotionally). Then use "showing writing" to 
make the reader feel like he or she is there watching what's going on 
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Screen Design for Rhetorical Interventions (CI) 
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Chunking with On-Line Guides 



Prompt: Choose a time your family was going through an important change 
(one that affected you emotionally). Then use "showing writing" to 
make the reader feel like he or she is there watching what's going on 
with your family. This should be a good-sized paragraph. 



:::::: 




Ideas 



What was the setting 
like? Describe specific 
sights, smells, or 
feelings. 



This is where the student types. 
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Screen Design for On-Line Guides (CG) 
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Full Treatment 




j Prompt: Choose a time your family was going through an important change 
(one that affected you emotionally). Then use "showing writing" to 
m ake, the reader fptel like, he. or is there, watching wha t's going on 
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I tiiant the reader to be feeling... 
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Figure 7 

Screen Design for the Full Treatment (CGI) 
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Control ' Program 



Prompt: 



Choose a time your family was going through an important 
change (one that affected you emotionally). Then use "showing 
writing" to make the reader feel like he or she is there watching 
what's going on with your family. This should be a good-sized 
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This is where the student types. 
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Screen Design for the Control Group (Control) 
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ii 'lerent in reflective writing are both difficult to learn and to teach. Second, 
reflective writing demands knowledge-transforming strategies to successfully 
complete it. Last, because the CAP Reflective domain is a high school level 
assessment, no direct instruction on the style of writing has occurred with middle 
school students, and thus, all learners qualify as **novices.'' 

The California Assessment Program scoring guide is a valid instrument to 
measure student performance^ It uses two scoring guides, one for style (sentence 
control and diction) and one for rhetorical effectiveness. Because this study 
investigates how writing quality, rather than basic skills, develops within a computer- 
guided environment, the latter assessment tool was the appropriate choice. As one of 
eight CAP domains all secondary students in California are expected to master, 
thorough statewide testing for four years served to validate the content of reflective 
writing and the reliability of the 6-point scoring guide (Appendix A). The scoring 
guide is also well-supported by holistic rubrics and anchor papers to train readers. A 
panel oi six middle and secondary English teachers served as readers for this study. 
The quality of the training guides aad the expertise of the raters served to control for 
the threat to validity due to instrumentation. 

Assessing Cognitive Awareness 

Bonk and Reynolds' {I990)lndex of Writing Awareness (see Appendix B) was 
used to assess students' metacognitive development. Adapted from Jacobs and 
Paris's Index of Reading Awareness (1987), the IWA uses a series of 20 questions, 
each followed by three possible responses. The three responses vary in value from 0 
to 2 points depending upon the degree of metacognition demonstrated by the 
students' understanding of their writing processes. Bonk and Reynolds have shown 
significant correlation between IWA and IRA scores for seventh and eighth grade 
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students (r= .35, p < .01 and r= .26, p < .05, respectively). Additionally, when the 
essays of low- and high-ability students (ability was determined by Metropolitan 
Achievement Test scores) Bonk and Reynolds found statistically significant 
differences in writing quality based upon ability (F = 17.68, p < .05). The significant 
correlation between an already established instrument (the Index of Reading 
Awareness) and tlie statistically significant differences that coincided with both 
standardized MAT scores and writing quality support the reliability and validity of 
the Index of Writing Awareness. 

Assessing Student Satisfaction in Using Computers 

Attitude toward compuier-use was assessed with Likert Bipolar Attitude 
Inventory items (see Appendix C). The measure was adapted from a questionnaire of 
items used repeatedly in the Metropolitan School District of Mount Vernon in Terre 
Haute, Indiana (Oilman, et al., 1991). Post-test reliabilities for the two questionnaires 
used C'About Computers" and "What I Can Do with Computers'') were computed as 
.90 and .94 respectively. The repeated use and high reliability of the measure 
recommends its use in an educational setting. 

Measuring Revision 

Faigely and Witte's Taxonomy of Revision (1981) (see Appendix D) was used 
to assess the number and degree of changes students made in revision. This measure 
has been used for similar studies by the originators of the instrument as well as Bonk 
and Reynolds in their 1990 study. Furthermore, Lauer and Asher list it as the sole 
measure for revision assessment in their Composition Research: Empirical Designs 
(1988). 
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Assessment of student revision was facilitated through the software itself. In 
the treatment, when students felt that they had completed their writing, they clicked a 
"finished?" button which took them to a ''revision" screen. At this point, the software 
saved an original version of the writing with which to compare the students' later 
revised draft. To expedite analysis, only Faigely and Witte's two basic categories of 
''surface" and "text-based" changes were assessed. The former tend to be spelling or 
typographical and do not affect the meaning of a text while the latter alter the 
meaning of a sentence or passage. 

Tracking Student Performance 

An internal mouse-click tracking system based upon previous research (Bonk 
& Reynolds, 1990; Bridwell et al, 1985; Williams & Dodge, 1993) was employed to 
collect information about how students used the software. First, the length of time 
students worked on a particular screen was recorded. Second, the tracking system 
recorded the type and frequency of on-line guides students selected. This information 
was used to ascertain the nature of the students' interplay with the optional guides. 
Third, word couniS of the students' final writing were recorded. Last, the students' 
paths through the software were tracked to learn how they made use of the options 
and buttons as well as to examine whether the recursive, non-linear pattern of expert 
writers emerged. 

Procedure 

Students at O'Farrell are scheduled into families with the goal of achieving 
ethnic balance. In terms of ability, however, student placement is completely 
random. Because the climate of the school promotes student collaboration, treatments 
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were randomly assigned to intact classes (hereafter referred to in the parlance of 
O'Farrell as **groups'')> rather than randomized individually across the six groups^ As 
the four treatments and one control would be used with six groups, the teachers were 
asked whether a particular class stood out as either high or low in ability. Although 
they felt that no clear distinctions existed among the groups, because a large majority 
of the students in group five had the limited skills that qualified them for Chapter One 
funding, it was decided to use the full treatment with group five and another randomly 
assigned group» It was decided that a pre-test writing would be used to control for 
selection as a threat to internal validity. In this way, if ability levels among groups 
posed a significant difference, an analysis of covariance would be employed. Table 6 
lists the assignments of group and software. 

Table 6 

Groups and Their Assigned Treatments 



Group Treatment 



One 


Chunking with inter\'entions (CI) 


Two 


Chunking with on-line guides (CG) 


Three 


Full treatment (CGI) 


Four 


Chunking only (C) 


Five 


Full treatment (CGI) 


Six 


Control 



Although the classes were taught by two different language arts teachers, only 
one instructor worked as the computer teacher. To control for the potential 
extraneous variable related to having two different English teachers presenting the 
introductory lessons, the instruction was scripted (see Appendix F) and supported by 
a computer-based presentation program. As it turned out, because of professional 
development activities, one of the teachers was unable to be present during *' 's initial 
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phase of instruction, so all preliminary lessons were taught by one teacher. Because 
all students had worked with both English teachers the previous semester, this seemed 
a fortuitous situation and promoted consistency of instruction among groups. 

These preliminary sessions included direct instruction on reflective writing 
which included definitions, example topics, and a sample paper communicated via 
handouts (see Appendix F) and liquid crystal projection of the computer-based 
presentation, ''Essay Means To Try''' (see Appendix G). Students were asked to 
come to the next session with a personal anecdote of an event in their family that had 
affected them emotionally. This anecdote was then used as the basis for a pre-test of 
the students' writing ability. This writing sample was word processed and blind- 
ranked by this study's author into categories of ''high, '^medium," and "low." These 
rankings were then presented to the students' teachers to verify the pre-test' s accuracy 
as a sample of the students' abilities. 

Additional data collection also preceded the experimental treatments. All • 
students completed pre- tests of the Index of Writing Awareness and the Likert scale 
assessing attitudes toward computer-use. Students were coached not to discuss their 
writing assignments with their friends in other groups until after the unit. 

Session # 1 

Each group received a short introduction on the particular software they were 
to use. Because the number of students exceeded the number of computers in the lab, 
several students in each group had to either wait or work at a computer in another 
room. Although many students had as much time as they needed to complete their 
reflective essays, a large minority was unable to finish within the period. Even 
though the goal was to allow all students unlimited time (including subsequent class 
meetings), the realities of a school schedule and lab availability impeded this goal and 
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students only had one period of approximately 80 minutes in which to write and 
revise their essays. Similarly, although a reading assignment or revising previous 
writing were suggested as activities for students who had finished, the routine of 
turning to computer games became the actual activity. The effects of these events are 
discussed in the last chapter. 

Session # 2 

Students were given a chance to become better acquainted with their treatment 
software in a second session, which occurred the following week. To integrate the 
experience with the students' normal curriculum, and thereby lessen the "specialness" 
of the experiment, the topic of the second guiding program related to the students' 
current focus on world events and social issues. Although this seemed like a 
reasonable choice at the time, its possible consequences are discussed in the last 
chapter. All the same procedures were followed for this session as the first (including 
the necessarily limited time-frame and the post- writing student activities). 

Follow-up Data Collection 

Once both essays were completed, students took post-tests of the Index of 
Writing Awareness and the computer satisfaction survey. Students then received an 
explanation of the experiment's purpose and were asked for feedback on the 
experience. 

Experimental Design 

This study followed a quasi-experimental 1 X 5 factorial design that examined 
the four versions of the treatment software and the control. Precautions used to limit 
threats to validity were a pre-test, which served as a covariate to diminish the 
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possibility that intact groups might be of unequal abilities, and the use of a control 
group to guard against the threat of maturation as students wrote the three essays (the 
pre-test and two treatment essays). 

Data Analysis 

Data was analyzed using StatView 4,0 software (Haycock et al., 1992) and the 
Statistical Package for the Social Sciences (SPSS) (Nie et aL, 1975). This section 
describes the analytic methods for each hypothesis. 

Research Question 1 : 

Does computer-guided writing software help students produce more 
rhetorically effective compositions? 

H(i) - Learners in the group using the full treatment (CGI) of 
computer guidance will write essays that rate higher on the 
California Assessment Program's Scoring Guide for Rhetorical 
Effectiveness than learners in the control group. 



Hyp# 


Ind Variables 


Dep Variables 


Measures 


Statistics 




Treatment 


Effectiveness 


CAP rating 


ANOVA/Scheff6 



This hypothesis was tested using ANOVA to determine whether differences 
exist between the means of the two groups. It was decided in advance that if the 
overall F - ratio were significant at the .05 confidence level, then the Scheff6 Test 
would be used to make all possible comparisons among means. The Scheff6 was 
chosen because it is a conservative statistical procedure, and, thus, if any statistically 
significant differences were found, the probability that they would not have occurred 
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by chance would be more likely. A confidence level of 5% was predetermined. If the 
contribution of the independent variable were less than that, the efficacy of the model 
would be of questionable merit. If the difference among groups were significant, 
however, ANCOVA would be run using the covariate pre-test writing and adjusted 
means calculated for each group. If the adjusted means still fell within the critical 
difference as determined by the Scheffe Test, then the results would be considered 
validated for the intact groups. In addition, planned analyses of learner ability were 
conducted to assess the shape of the relationship betv^'een high ability users and 
computer-guided writing. 

Research Question 2 : 

Are there definable types of computer guides that promote specific expert-like 
performances in novices? 

H(2) - Learners in the treatment group with only structural 
chunking (C) will produce writings that rate higher on the 
California Assessment Program's Scoring Guide for Rhetorical 
Effectiveness than learners in the control group. 

H(3) - Learners in the treatment group with on-line guides and 
structural chunking (CG) will produce writings that rate higher 
on the California Assessment Program's Scoring Guide for 
Rhetorical Effectiveness than learners in the treatment group 
with only structural chunking (C). 

H(4) "Learners in the treatment group with structuml chunking 
and interventions (CI) will achieve higher scores on the Index 
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of Writing Awareness than learners in the treatment group 
using only structural chunking (C), 

H(5) Learners in the group with the full treatment of 
computerized guiding (CGI) will demonstrate more expert 
characteristics than learners in all other treatment groups as 
measured by higher scores in each of the following three 
assessments: California Assessment Program's rhetorical 
ratings. Index of Writing Awareness measures, and the degree 
of revision as determined by the Taxonomy of Revision, 



Hyp # Ind Variables Pep Variables Measures Statistics 



H(2) 


Chunking 


Effectiveness 


CAP rating 


AN0VA/Scheff6 


H(3) 


On-line guides 


Effectiveness 


CAP rating 


ANQVA/Scheff6 


H(4) 


Intervention 


Cog. awareness 


IWA 


AN0VA/Scheff6 


H(5) 


Full treatment 


Expert characteristics 


CAP rating 

IWA 

Revision 


MANOVA 



Hypotheses 2 through 5 were tested using the ANOVA and Scheff6 multiple 
comparison procedures with verification by ANCOVA and adjusted means. 
Hypothesis 5 was tested with the Software Package for the Social Sciences (SPSS) 
MANOVA procedure to validate any differences among the variables suggested by 
the ANOVA and Scheffd. 

Research Ouestion 3 : 

What is the interaction betv/een learner ability and computer support? 
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H(6) - Lower ability students in the treatment group with the 
full treatment of computerized guiding (CGI) will demonstrate 
more expert characteristics than low ability students in the 
control group as measured by higher scores in both Califomia 
Assessment Program's rhetorical ratings and Index of Writing 
Awareness measures* 



Hyp# 


Ind Variables 


Dep Variables 


Measures 


Statistics 


H(6) 


Full treatment 


Expert characteristics 


CAP rating 


MANOVA 




Low ability level 




IWA 





Hypothesis 6 was tested along the lines described for hypothesis 5, using the 
SPSS MANOVA procedure to determine differences among the variables. Based 
upon the work by McCutchen (1988) and Bereiter and Scardamalia (1987), 
instructional strategies targeting novice writers must force a change in the writer's 
procedure. The collected data was used to measure whether a change took place. 
Similar data were also collected from high ability students to assess whether an 
interaction exists between these learners and a high level of computer guidance. 

Research Question 4 : 

Is there a relationship between computer support and a student's level of 
satisfaction in using computers as a writing tool? 

H(7) - Students in the full treatment group (CGI) will express 
greater levels of satisfaction in using computers than those in 
the control group. 
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Hyp # Ind Variables 



Pep Variable 



Measure 



Statistics 



H(7) Full treatment 



Attitude 



Liken scale 



AN0VA/Scheff6 



This hypothesis was tested by the ANOVA and Scheff6 planned multiple 
comparison procedures. Research suggests that students generally like working with 
computers (Montague, 1990). This question attempted to ascertain differences in 
attitude when students work within a computer-guided writing environment. The 
same measures were taken from all students so that additional insights might be 
gained concerning the interaction between effectiveness and attitude. 
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CHAPTER IV 



RESULTS 
Chapter Outline 

This chapter is organized into five main sections. First, the processes for 
obtaining statistical data are presented. Second, descriptive evidence from the 
dependent variables are reported. Third, each of the hypotheses are tested. Fourth, 
additional findings are reported. Last, the results of the study are summarized. 

Description of the Processes Used to Obtain the Results 

Before interpreting the descriptive statistics, a brief overviev^ of the essay- 
scoring and data-analysis processes is appropriate, beginning with a description of the 
essay rating process. First, all 385 essays (pre-tests and two treatments) were 
prepared for the rating session. Students names were taped over and papers from all 
groups and all treatments were shuffled together. In this way readers could not 
identify student writers or computer treatment groups. Next, after a thorough training 
session using CAP anchor papers and the scoring guide, all essays were rated by at 
least one reader. After the reading session, to assess reliability across readers, the 
author of this study randomly selected and rated 10 papers from each reader. 
Correlation coefficients between each of the six readers and the author showed that 
five of the readers were between .812 and .896. This correlation was considered 
acceptable. After analyzing the papers of the sixth reader (r= .195), it was 
determined that the chief differences involved papers given a score of 1. All papers 



77 

scored 1 were then rated a second time by the present author who had no knowledge 
of the treatment group in which a particular essay belonged. 

Also, before testing the hypotheses, a word should be said concerning the 
process of data analysis. Because treatments were assigned to intact classes, 
ANCOVA was conducted to statistically factor out the inherent variability among 
groups. The analysis of covariance conducted on SPSS supported the ANOVA 
results calculated by StatView. Further comparison between unadjusted and adjusted 
means found that all adjusted scores still fell within the critical differences for each 
group. Because the apparent effect of innate ability had little impact on differences 
among groups, it was considered acceptable to use the ANOVA tests on StatView. 
This choice was made because StatView offered the flexibility to conduct Scheff^ 
multiple comparisons among groups and thus gain greater precision in terms of 
determining statistical differences between specific versions of the treatment. 
Therefore, the ANOVA / Scheff6 procedures were employed to test the hypotheses of 
this study and are listed throughout this section. 

Overview of the Descriptive Statistics 

To begin the analysis of the descriptive statistics. Figure 9 graphically shows 
the increased frequency of higher mean scores when students wrote with the suppoit 
of guiding software. Of particular interest are the fewer number of essays scored 
"one" and the larger number of "three" essays in the treatment groups. 

In addition to the frequency histogram, examination of the mean scores (Table 
7) among pre-test and treatment essays also confirms the advantage offered by 
computer guidance. Notice that essay 1 and essay 2 means are approximately equal 
and considerably higher than tlie unguided pre-test means. 
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I .gure 9 

Frequency of Scores for all Three Writings 



Table 7 

Descriptive Statistics for Dependent Variables 






Count 


Mean 


Std. Dev. 


Std. Error 


Minimum 


Maximum 


CAP (pre) 


139 


1.532 


.792 


.067 


0.000 


4.000 


CAP#l 


112 


2.188 


.822 


.078 


0.000 


4.000 


CAP #2 


134 


2.172 


1.000 


.086 


0.000 


5.000 


IWA-Prc 


121 


27.182 


4.368 


.397 


11.000 


35.000 


IWA - Post 


137 


26.686 


5.107 


.436 


13.000 


37.000 


Attitude Pre 


135 


51.437 


7.096 


.611 


28.000 


64.000 


Altitude Post 


135 


51.563 


5.579 


.480 


40.000 


63.000 
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On the other hand, mean Index of Writing Awareness scores represented on 
the same table actually decreased, with student satisfaction in using computers 
remaining virtually unchanged during the three-week treatment period. Before testing 
the individual hypotheses, further detail conceming mean scores across groups 
provides interesting data. 

Table 8 presents mean scores categorized by the software version used. First, 
notice that adjusting the means with the covariates did httle to alter the relationships 
among groups. Next, note that a comparison between pre- test writing scores and the 
essay 1 scores shows that all treatment groups demonstrate increases of from .634 (C) 
to .907 (CI). The control group, however, demonstrates a difference of only .211. 



Table 8 

Means Table for Essavs bv Group 



Group 


Writing 


Count 


CAP Mean 


Adj. Mean 


Std. Dev. 


Chunking 


Pre-test 


25 


1.583 




.748 


(C) 


Hssay 1 


22 


2.217 


2.187 


.685 




Essay 2 


22 


2.292 


2.282 


.950 


Intervention 


Pre-test 


24 


1.320 




.711 


(CI) 


Essay 1 


20 


2.227 


2.357 


.607 




Essay 2 


23 


2.045 


2.115 


.964 


On -Line Guides 


Pre-test 


24 


1.625 




.881 


(CG) 


Essay 1 


23 


2.500 


2.450 


.902 




Essay 2 


24 


2.739 


2.709 


1.042 


Full Treatment 


Pre-test 


48 


1.625 




.815 


(CGI) 


Essay 1 


27 


2.333 


2.243 


.961 




Essay 2 


45 


2.089 


2.059 


.996 


Control 


Pre-test 


18 


1.389 




.778 




Essay 1 


20 


1.600 


1.660 


.598 




Essay 2 


20 


1.700 


1.7.30 


.801 
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Thus, mean scores reveal that students using variations on the guided-writing 
software wrote essays that scored higher on the CAP Scoring Guide than students 
who did not The difference increase between the chunking (C) and control scores is 
particularly revealing in that the programs were identical except that the chunking 
program divided the instructions up over six screens. That students raised their scores 
almost half a grade simply because they were prompted to address one movement of 
the writing at a time would seem to support the notion of schematic chunkmg. This 
movement is graphically depicted in Figure 10. Here the guided treatment essays 
show higher mean scores than the pre-test. Further analysis of the graph reveals a 
pattern where the on-line guides (CG) treatment group achieved the highest mean 
scores on essays 1 and 2 and the control group earned the lowest. 

3 T 




1 H H 1 1 1 

Control C CI CG CGI 

Treatments 

.««..,..... Prc-Tcst Essay 1 Essay 2 

Figure 10 

The Effects of Treatments on CAP Mean Scores 
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Thus, the descriptive evidence suggests that an advantage exists when students 
compose using computer-guided writing software. In order to determine whether this 
difference is significant or not, the hypotheses must be tested and inferential statistics 
conducted. 

Testing the Hypotheses 

The first hypothesis of this study addressed the question of whether computer- 
guided writing programs help students write more rhetorically effective compositions. 
Hypotheses 2 through 5 then serve as ancillary suppositions that attempt to assess the 
effects of different types of computer guidance. Because the answer to the first 
question makes the following hypotheses either interesting or irrelevant, this section 
analyzes the first hypothesis in the greatest detail. It then examines the ancillary 
hypotheses concerning types of guides will then be examined (H(2) - H(5)), followed 
by investigation into the interaction between treatment and ability level (H(6)) and, 
lastly, it assesses student attitude toward using computers was assessed (H(7)). The 
hypotheses are listed with each analysis for reference. 

Computer Guidance Significantly Improves 
Student Writing . 

H(i) " Learners in the group using the full treatment (CGI) of 
computer guidance will write essays that rate higher on the 
California Assessment Program's Scoring Guide for Rhetorical 
Effectiveness than learners in the control group. 



In testing this hypothesis, the following procedure was followed. First, data 
were examined for the writing pre-test and each of the two treatment essays. 
Beginning with a pre-test that demonstrated no statistically significant differences 
among groups (F = .864, p = ,4873), an examination of the ANOVA Tables for essay 
#1 and #2 (Tables 9 & 10) reveals that use of computer guidance did make a 
statistically significant difference among the treatment groups at the .05 confidence 
level. 



Table 9 

ANOVA Table for Essay #1 

DF Sum of Squares Mean Square F- Value P-Value 

Treatment 4 9.486 2,371 3.869 .0056 

Residual 107 65.577 ,613 



Table 10 

ANOVA Table for Essav #2 





DF 


Sum of Squares 


Mean Square 


F-Value 


P-Value 


Treatment 


4 


12,860 


3.215 


3.451 


,0103 


Residual 


129 


120.192 


.932 







Because both guided essays show probability values significant beyond the 
predetermined confidence level, the Scheff^ Test was then conducted to ascertain 
differences among groups using versions of the treatment. Table 1 1 provides 
statistical evidence that supports the hypothesis for essay 1, but not for essay 2, 
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Table 11 

Scheffe Tes t for the Full Treatment and the Control 



Groups 


Essay # 


Mean Diff. 


Crit. Diff. 


p - Value 


Full treatment. Control 


1 


.733 


.724 


.0454 




2 


.389 


.811 


.6907 



Thus, the particular combination of chunking, interventions, and on-line 
guides (CGI) has a mixed impact when compared to the control group and therefore 
the research hypothesis could was rejected outright. 

Different Types of Guides Show no Significant Effect 

To better understand the nature of computer-guided writing and gain potential 
insight on the differences among types of guides, the following hypotheses were 
established to compare softwai*e versions featuring variations on the model. 

Structural Chunking has a Non-Significant Effect 

H(2) - Learners in the treatment group with only structural 
chunking (C) will produce writings that rate higher on the 
California Assessment Program's Scoring Guide for Rhetorical 
Effectiveness than learners in the control group. 

The same procedure was used to test the second hypothesis. In this case, as 
revealed in Table 12 for essay 1, the treatment with only structural chunking did not 
demonstrate significance at the predetermined confidence level of 5%. The research 
hypothesis was rejected. 
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Table 12 

Scheff6 Test for Structural Chunking 



Groups 


Essay # 


Mean Diff. 


Crit. Diff. 


p - Value 


Chunking, Control 


1 


.617 


.750 


.1639 




2 


.592 


.913 


.3971 



On-Line Guides Show a Limited Effect 

H(3) - Learners in the treatment group with on-line guides and 
structural chunking (CG) will produce writings that rate higher 
on the California Assessment Program's Scoring Guide for 
Rhetorical Effectiveness than learners in the treatment group 
with only structural chunking (C). 

The ANOVA an 1 Sc'ieff6 Test again served as the planned comparisons to 
analyze the fourth hypothesis. In this case, the use of on-line guides (CG) versus 
structural chunking alone (C) failed to make a statistically significant difference as 
shovv-n in Table 13, thus compelling the rejection of the research hypothesis. 



Table 13 

Scheff€ Test for On-Line Guides 



Groups 


Essay # 


Mean Diff. 


Cril. Diff. 


p - Value 


On-Linc Guides / Chunking 


1 


.283 


.750 


.8445 




2 


.447 


.880 


.6413 



Rhetorical Interventions Fail to Increase Student Cognition 

H(4) -Learners in the treatment group with structural chunking 
and interventions (CI) will achieve higher scores on the Index 
of Writing Awareness than learners in the treatment group 
using only structural chunking (C). 

As expected from the descriptive evidence presented in the opening of this 
chapter (Table 8), one would assume that the use of rhetorical interventions had no 
positive impact on student cognition. Table 14 reports the IWA pre- and post-test 
means categorized by version of treatment software. Of particular interest are the 
consistently lower scores on the versions employing interventions (CI and CGI). 

Table 14 

Means Table for Index of Writing Awareness Scores 



Group 


IWA 


Count 


Mean 


Std. Dev. 


Chunking (C) 


Pre-test 


22 


26.955 


4.100 




Post-test 


24 


26.875 


4.485 


Intervention (CI) 


Pre-test 


25 


25.680 


5.226 




Post-test 


22 


23.955 


5.980 


On-Line Guides (CG) 


Pre-tcsl 


21 


27.905 


4.538 




Post-test 


22 


29.182 


3.621 


Full treatment (CGI) 


Pre-test 


36 


27.083 


3.909 




Post-test 


47 


26.617 


5.566 


Control 


Pre-tcsl 


17 


29.000 


3.623 




Post-test 


22 


26.864 


3.968 



Figure 1 1 graphically depicts the tendency for post-test means to be lower 
than pre-test means. In fact, in all but one case (CG), the treatments had a negative 
impact on students' understanding of their own writing process. The planned 
ANOV A was conducted on the IWA post-test and revealed a p - value significant at 
the .05 level. Further analysis using the Scheff6 Test supported the assumptions 
based upon Figure 1 1 that the difference between the CI and CG groups was the 
source of the significance (F = 3.081, p = .0183). 




Because the only distinguishing feature between the C and CI groups was the 
addition of interventions, these two groups served as the focus for hypothesis 3. 
Table 15 demonstrates the p - values for the hypothesized diffeit:nces. 
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Table 15 

Scheff6 Test f or Rhetorical Interventions 



Groups 



rWA 



Mean Diff , 



Crit Diff, 



p - Value 



Chunking, Interventions 



Pre-test 



A215 



3.953 



,9063 



Post-test 



-2,920 



4,572 



.4123 



Table 15 shows that the difference was not statistically significant. However, 
the negative impact of these interruptions in the learner's process should be noted. In 
fact, for all but one group (CG), interventions had a negative impact on students' 
mean scores on the Index of Writing Awareness. Clearly, the research hypothesis 
was rejected. 

The Full Treatment Doesn't Contribute to Expert Traits , 



H(5) " Learners in the group with the full treatment of 
computerized guiding (CGI) will demonstrate more expert 
characteristics than learners in all other treatment groups as 
measured by higher scores in each of the following three 
assessments: California Assessment Program's rhetorical 
ratings. Index of Writing Awareness measures, and the degree 
of revision as determined by the Taxonomy of Revision, 



The measures used to test this hypothesis had to be altered because students 
engaged in a minimal amount of revision. Evidence of this was available via the 
computer-tracking system. Random sampling of 10 "high" ability students' papers 
yielded a mean of less than two surface-level changes per paper. Rather than abandon 
the notion of an array of **expert characteristics," two other measures were used to 
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supplant the Taxonomy of Revision scores and supplement the CAP and IWA scores. 
These were the number of times a student returned to earlier sections (thus 
demonstrating the expert's tendency to review earlier passages) and the length of time 
spent on the final reflection. Thus the revised hypothesis reads: 

H(5) - Learners in the group vvith the full treatment of 
computerized guiding (CGI) will demonstrate more expert 
characteristics than learners in all other treatment groups as 
measured by higher scores in each of the following four 
assessments: CAP ratings, Index of Writing Awareness 
measures, the number of times a student returns to earlier 
sections, and the length of time spent on the final reflection. 

With this adjustment made, examination of Means Table 16 below shows that 
the full treatment did not achieve superiority in any of the five dependent variables. 



Table 16 

Mean Scores for Measures Assessing Expert Characteristics 



Group 


Essay 1 


Essay 2 


IWA 


Returning 


Reflection 


Chunking (C) 


2.217 


2.292 


26.88 


5.9 


291.78 


Intervention (CI) 


2.227 


2.045 


23.96 


3.2 


451.13 


On-Line Guides (CG) 


2.500 


2.739 


29.18 


2.5 


268.22 


Full Treatment (CGI) 


2.333 


2.089 


26.62 


2.1 


337.59 



To complement the previous graphs depicting the differences among treatment 
groups on variables, similar figures represent the added dependent variables of 
"Returning'' and "Reflection." Figure 12 shows that the relationship between 
treatment and the mean number of times students "returned'' to previous passages 
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favors the chunking (C) version as opposed to those featuring on-line guides. Figure 13 
conversely supports the rhetorical intervention strategy as promoting the length of time in 
seconds students spent reflecting during the "Final Awareness" section. 



# of times 
students 
"returned" to 
earlier sections 




Treatments 

Figure 12 

"Retuming" Movements Versus Treatments 



Reflection time 
(seconds) 




Treatments 

Figure 13 

Time Spent on Reflection Versus Treatments 
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Even though the mean scores indicated that the CGI version provided no 
consistent advantage over the other versions, Analysis of Variance was conducted on 
all variables to ascertain whether any statistically significant differences existed 
among groups. As already determined by the Scheff6 comparison for the first 
hypothesis, the full treatment (CGI) was significantly different from the control (p = 
.0454), but did not, however, demonsti'ate a significant difference when compared to 
the other treatments. Regarding the IWA, Table 16 indicates the full treatment 
produced lower IWA scores than all but the program featuring interventions. As to 
assessing the mean number of times students returned to previous sections of their 
writing. Table 16 reveals that students using the full treatment returned less frequently 
than any other group. Lastly, students who used the full level of treatment did take 
more time to reflect than their peers in all other groups except for those writing with 
the intervention program. This difference, however, was not significant. Because the 
full treatment (CGI) was not superior to any other version in any of the categories, the 
M ANOVA procedure was not needed to test the hypothesis, and the research 
hypothesis was rejected. 

The Full T reatment has no Effect on 
Low Abilitv Students, 

H(6) -- Lower ability students in the treatment group with the 
full treatment of computerized guiding (CGI) will demonstrate 
/ more expert characteristics than low ability students in the 

control group as measured by higher scores in both California 
Assessment Program's rhetorical ratings and Index of Writing 
Awareness measures. 
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The planned test for the sixth hypothesis was MANOVA. Examination of 
means Table 17, however, reveals that no consistent pattern of improvement exists 
from the control to the treatment group. In fact, except for the pre-test and essay 1, 
the control achieved higher mean scores on all the measures. For this reason, no 
MANOVA procedures were conducted. Other analytical comparisons were made for 
interaction and these findings are discussed in the next section. As for the full 
treatment's effect on developing expert characteristics in low ability students, the 
research hypothesis was rejected. 

Table 17 

Mean Scores for Low Ability Students 



Group 


Measure 


Count 


Mean 


Std. Dev. 


CGI /Low 


Pre-Test 


17 


1.176 


.393 


Control / Low 


Pre-Test 


3 


1.000 


0.000 


CGI /Low 


Essay 1 


6 


1.500 


.548 


Control / Low 


Essay 1 


6 


1.000 


0.000 


CGI /Low 


Essay 2 


15 


1.667 


6.17 


Control / Low 


Essay 2 


4 


1.750 


.957 


CGI /Low 


IWAPre 


9 


24.33 


3.08 


Control / Low 


IWAPre 


4 


24.75 


3.20 


CGI /Low 


IWA Post 


17 


23.71 


5.53 


Control / Low 


IWAPost 


5 


24.80 


5.54 



The Full Treatment Doesn't Improve Satisfaction 

H(7) - Students in the full treatment group (CGI) will express 
greater levels of satisfaction in using computers than those in 
the control group. 
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Descriptive evidence presented at the beginning of this chapter reported the 
pre- and post-test means of the attitude assessment as 51.437 and 51.563. One would 
not, therefore, expect to see major changes in attitude between groups. Table 18 
reveals negligible differences between pre- and post-test data within all groups except 
for those students using the chunking version (C). 

Table 18 

Means Table for Compu ter-Use Satisfaction 



Group 


Attitude Survey 


Count 


Mean 


Std. Dev. 


Chunking (C) 


Pre-test 


23 


50.130 


7.294 




Post-test 


24 


52.000 


5.823 


Intervention (CI) 


Pre-test 


25 


49.600 


6.178 




Post-test 


22 


49.591 


5.933 


On-Line Guides (CG) 


Pre-test 


21 


49.905 


8.012 




Post-test 


22 


49.500 


5.934 


Full treatment (CGI) 


Pre-test 


46 


53.761 


7.053 




Post-test 


46 


53.217 


4.447 


Control 


Pre-test 


20 


51.500 


6.143 




Post-test 


21 


51.667 


5.969 



Even though the control had a lower mean score than the full treatment group 
(CGI), the table shows that the groups' attitudes did not change over the course of the 
treatment, but maintained the pre-test difference. The planned ANOVA was 
conducted and did report a significant difference among groups (F = 2.608, n = 
.0386). However, the Scheffd Test failed to show any statistically significani 
differences among groups. Bee mse no positive increase in student attitude occurred. 
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the research hypothesis was rejected. Inferences that can be drawn from this data are 
limited and are brought up in the discussion. 

Additional Analyses 

Descriptive findings pointed to several areas for further analysis. The 
following section will explore statistically significant effects involving the software 
version with chunking and on-line guides (CG); the interaction between guided 
writing and ability; and correlations among the Index of Writing Awareness and the 
length and quality of writings, 

On-Line Guides as Effective 
Guided- W riting Software 

Although the full treatment (CGI) only achieved significance for one of the 
two essays, the treatment featuring structural chunking and on-line guides (CG) made 
a statistically significant difference when compared with the control group for both 
essay 1 and essay 2, Table 19 presents the descriptive data. 

Table 19 

Means Table for On-Line Guides 



Writing 


Count 


CAP Mean 


Std. Dev. 


Pre-test 


24 


1.625 


.711 


Essay I 


20 


2.500 


.607 


Essay 2 


23 


2.739 


.964 



Figure 14 recalls the earlier depiction from the beginning of this chapter to 
convey the consistent superiority of the treatment featuring on-line guides. Planned 
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Scheff6 multiple comparisons further verify the benefit of this treatment. When 
compared to the control, the treatment version with on-line guides (CG) consistently 
achieves statistical significance at the .05 level as shown in Table 20. Thus, dividing 
the movements of a reflective essay into semantic chunks and offering students 
optional assistance in the form of examples, ideas, and strategies is an effective 
instructional strategy for improving the rhetorical quality of student writing. 




1 



+ 



CG 



Control C 

in»iiiiimi«Pre-Test 

Fi gure 14 

The Superiority of On-Line Guides Based Upon CAP Means 



1 

CI 

Treatments 
-««*««««Essay 1 



1 

CGI 

■Essay 2 



Table 20 

Scheff6 Test for On-Line Guides 



Treatments 


Essay # 


Mean Diff. 


Cril. Diff. 


p - Value 


On-Line Guides / 


1 


.900 


.776 


.0135 


the Control 


2 


1.039 


.922 


.0179 
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In terms of the other key dependent variable, the Index of Writing Awareness, 
the program featuring on-line guides (CG) gains further support. First, reexamination 
of the line graph displaying IWA scores across treatment groups (Figure 15) 
highlights the positive effect of on-line guides as well as the statistically significant 
difference between it and the treatment featuring interventions (CI). 




23 " 

22 -I 1 1- 1 \ 

Control C CI CG CGI 

Treatments 

Figure 15 

The Superiority of On-Line Guides Based Upon IWA Means 

Specifically, the ANOVA conducted on the IWA post-test revealed an F - 
value of 3.081 and a p - value of .0183. The Scheff6 analysis further pointed up a 
statistically significant difference between the on-line guides group (CG) and that 
using interventions (CI). The mean difference of 5.23 between the groups achieved a 
p - value of .0109. Although this is not as persuasive as a comparison between a 
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treatment and a control, the positive effect of on-line-guides on student metacognition 
is worth noting. 

Last, analysis of the computer tracking system allows some early trends to be 
noted about actual student use of on-line guides. The recorded mouse-clicks of 
students using both the CG and CGI models reveal some interesting findings listed in 
Table 21. 



Table 21 

Descriptive Data for On -Line Guides 





Mean 


Std. Dev. 


Std. EiTOr 


Count 


Min. 


Max. 


Examples 


5.882 


4.493 


.629 


51 


0.000 


16.000 


Ideas 


14.216 


18.920 


2.649 


51 


0.000 


87.000 


Strategies 


1.412 


3.106 


.435 


51 


0.000 


18.000 



Though by no measure conclusive because of the small sample and the erratic 
nature of the guides' use, this early information on student interaction reveals 
interesting evidence. Table 22 shows that the mean use of the "example" button 
(5.909 and 5.862) may suggest a correspondence between the number of chunks (6) 
and the use of the examples. Pattems varied somewhat betw<^^n groups regarding 
**ideas," although both groups made negligible use of the **strategies." As stated, little 
should be inferred by such early and unposited findings, but the fact that some 
pattems and consistency exist bodes well for future research. 
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Table 22 

Mean Use of On-Line Guides 



Treatment 



Mean 
**Examples' 



Mean 
**Ideas^ 



Mean 
"Strategies" 



On-line guides (CG) 
Full treatment (CG I) 



5,909 



5.862 



10.545 



17.00 



1.273 



1.517 



To summarize, then, the treatment featuring a combination of stnictural 
chunking and on-line guides produces a statistically significant effect in temis of 
rhetorical effectiveness. In a limited, three- week treatment period, with no additional 
instruction between essays, the group using or-Une guides raised their mean score 
over one full point. The treatment also had a significant effect on the Index of Writing 
Awareness scores when compared to the lowest group. Thus, computer-guided 
writing software configured with chunking and guides can be said to contribute to 
better writing and may promote advanced cognition. 

Interactions Between Guided Writing and Ability 

Although the sixth hypothesis, addressing interaction effects between 
treatment and ability, showed no differences between the CGI group and the control, 
further analysis of this interaction was conducted. Of specific interest was whether 
any interactions might exist among versions of the treatment. Figures 16 and 17 
display the interaction for essay 1 and 2. 

Of particular interest in Figure 16 for essay 1 is the fact that the treatment with 
chunking and on-line guides (CG) acted as something of an equalizer of ability. In 
order to further investigate the interaction, an ANOVA was run limiting ability to see 
if the main effect for treatments was significant. Table 23 shows findings that support 
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the effectiveness of the treatments when ability is controlled for. Of interest in the 
table is that when the strong effect of ability is limited, the treatments still produce a 
statistically significant difference among groups. 
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Ability 

Figure 16 

Interaction Effect Between Treatments and Ability for Essay 1 



Table 23 

ANOVA Table for Treatment and Ability on Essay 1 





DF 


Sum of Squares 


Mean Square 


F-Value 


P-Value 


Treatment 


4 


6.280 


1.570 


2.947 


.0308 


Ability 


1 


11.251 


11.251 


21.120 


<.0001 


Treatment * Ability 


4 


6.134 


1.533 


2.878 


.0337 


Residual 


43 


22.907 


.533 
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Figure 17 gives the same information regarding treatment essay 2. Of note 
here is the more regular pattern of interaction. With the control offering no positive 
effect tor either low or high ability students, again the treatment with on-line guides 
and chunking (CG) benefited both high and low ability levels. Also, as in the figure 
for essay 1, no Unes move in a strongly opposite slope, suggesting that, in general, the 
treatments do not have contrary effects on different ability levels. 



Figure 17 

Interaction Effect Between Treatments and Ability for Essay 2 

Furthermore, the case against rhetorical interventions is strengthened when the 
ability / treatment interaction is plotted on a graph. Notice that for low ability 
students the control program proved more effective for essay 2 tlian all treatment 
programs except for that featuring on-line guides (CG). Although these differences 
are non-significant, they may suggest trends that would increase understanding of 
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how the effects of user ability and guided- writing interact. One trend seems to 
suggests that the forced constraints of interventions and chunking without on-line 
support hinder wiiting perfonnance for low ability students. Thus, although an 
ANOVA calculating the interaction between the effects of treatment and ability was 
not significant (F = .950, p = .4419), Figure 18 illustrates, using a different dependent 
variable (the IWA), that again interventions promote a negative trend for low ability 
students. 




Ability 

Figure 18 

Interaction Effect Between Ability and Treatment on IWA Scores 

Thus, to conclude this analysis of the interaction between treatments and 
abilities, there w?.5 a non-significant trend toward an interaction for essay 1, but not 
for essay 2. Also, though not significant, the subtleties in slope suggest that low 
ability students do worse when they encounter additional cognitive demands while in 
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the process of writing as evidenced by the negative impact of the pop-up interventions 
and unsupported chunking. 

The Index of Writing A wareness as 
an Indicator of Ability 

Because one of the hypotheses investigated how computer-guided writing 
might affect low ability students, a sorting by thirds of the pre-treatment essay was 
conducted. Papers were categorized as low, middle, and high according to three main 
criteria: first, in harmony with the profile of novice writers, the degree to which 
writers moved away from a simplistic, linear re-telling of an event; second, the 
specificity of detail used in description; and last, the mastery of written conventions 
evidenced in the text. The Index of Writing Awareness (Bonk & Reynolds, 1990) has 
shown a high correlation to writing ability and thus could be used as a similar sorting 
tool. With this in mind, additional analyses were conducted to assess the measure's 
accuracy at predicting student ability. 

Initial analysis of the descriptive evidence shows a dkect relationship between 
more traditional assessment methods and the IWA. Mean scores on Table 24 
illustrate this relationship. Of particular interest is the near across-the-board 
correspondence among the three other, more traditional, dependent variables and 
themean IWA scores. For example, at least a 4 point difference in mean scores exist 
between low and high ability students in each treatment group. Similarly, tliree of the 
five groups show differences of at least 1 point on the CAP measure for essay 1 while 
four groups demonstrated this 1 point margin for essay 2. Commensurate differences 
exist for all five groups when the number of words are considered. In fact, for the 
two groups with the greatest difference (Chunking and Intervention) in number of 
words across ability levels we also see differences of almost 2 points on tht CAP 
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score for essay 2. Certainly, some correspondence exists between Index of Writing 
Awareness scores and these more traditional types of assessment. 

Table 24 

Mean Scor es on the IWA. Essavs. and Number of Words 



Group 


IWA 


Essay 1 


Essay 2 


# of Words 


Chunking / High 


31.00 


3.200 


3.200 


497.20 


Chunking / Low 


26.00 


1.500 


1.333 


218.00 


Intervention / High 


30.25 


2.250 


3.250 


450.25 


Intervention / Low 


23.63 


1.857 


1.333 


217.67 


On-Line Guides / High 


29.17 


2.667 


3.500 


541.67 


On-Line Guides / Low 


24.75 


2.667 


2.000 


352.20 


Full Treatment / High 


28.36 


3.250 


2.727 


496.30 


Full Treatment / Low 


24.33 


1.500 


1.667 


293.50 


Control / High 


31.00 


2.000 


1.750 


369.75 


Control / Low 


24.75 


1.000 


1.750 


293.50 



As stated, previous work by the originators of the instrument (Bonk & 
Reynolds, 1990) showed a high correlation between writers' ability and their IWA 
scores. Additional correlations conducted in this study further support the IWA as a 
useful tool. The matiix in Table 25 notes highly significant correlations between 
several key dependent variables. 

Clearly, high probabilities correlate the Index of Writing Awareness and other 
more traditional forms of measurement as word count and writing quality. The 
indication is that both composition quality and length show a strong positive 
correlation with a student's understanding of the writing process. This correlation 
further supports the efficacy of the Index of Writing Awareness as an alternative 
assessment determining writing ability among middle school students. 
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T; ble 25 

Correlation Matrix for the IWA and Other Measures 



Essay #1 Essay #2 IWA -Pre IWA -Post # of Words 



Essay (pre) .440** 


.161* 


.445** 


.410** 


.180 


Essay #1 


.359* 


.172 


.308* 


.269* 


Essay #2 




.329* 


.382** 


.351* 


IWA -Pre 






.523** 


.285* 


IWA - Post 








.316* 



ff of Words 

*p<.05 **p<,001 

Tracking Information 

As demonstrated in Table 25 the mean score for essay 2 and the number of 
words written also show a high correlation (r= .351, p = .0025). Although one might 
make quick judgments that essays were scored on length, as opposed to quality, 
examination of the CAP Scoring Guide (Appendix A) reveals no mention of length 
other than in terms of "development" Typically, a long paper by a novice writer 
springs from the automatic, non-conscious fluency inherent in the knowledge-teller 
model (Bereiter & Scardamalia, 1987; McCutchen, 1988). Figures 19 and 20 show 
the higher mean scores of the guided eiisays and their corresponding increases in word 
count. The correlation between word count and quality suggests that perhaps the 
guidance offered by the treatment programs helps steer students toward more 
developed, non-linear writing as described in the scoring guide. 
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310 + 

300 

Control 



CI 



CG 



CGI 



Treatments 

Figure 20 

Number of Words in Essay 2 Versus Treatments 
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Examination of Table 26 highlights the dominance of the two treatments that 
did not use interventions. In each of the two dependent variables, the CC and C 
treatments surpass the CI and CGI versions. The consistency of this pattern may 
suggest th?' the C and CG students had more time to write their compositions because 
they were not interrupted to answer rhetorical questions. If word count is so highly 
correlated to quality, prompts that slow the process seem detrimental. Although not 
convincing, these data again serve to recommend the treatment using on-line guides 
and reveal the negative effects of rhetorical interventions. 



Table 26 

Ranking of Writing Programs by Word Count and CAP Score 

Computer Guiding M # Words Rank M Essay #2 Rank 

On-Line Guides (CG) 455.818 1 2.739 1 

Chunking (C) 406.071 2 2.292 2 

FuU treatment (CGD 399.933 3 2.089 3 

Interventions (CI) 350.136 4 2.045 4 

Control 331.111 5 1.700 5 



Thus, to conclude, it seems that one feature of an effective guided-writing 
program is to promote longer, more sophisticated compositions by leading the novice 
through the writing process with optional guides for assistance. 

Summary of Results 

Computer-guided writing software provides a statistically significant positive 
effect to novice writers in terms of rhetorical effectiveness. Although the full 
treatment of computer guided-writing (CGI) proved effective on only one of the 
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essays, the combination of structural chunking and on-line guides (CG) consistently 
gave students a statistically significant advantage over students in the control group. 

The more specific hypotheses relating to types of computer guides, however, 
failed to show significant differences. The deeply rooted characteristics of cognition 
and affect did not undergo any significant change over the three-week treatment 
period. Additionally, an analysis of the interaction among treatments and ability 
revealed that when ability is controlled for, the treatments still demonstrate 
statistically significant differences among groups for one of the essays and mdss 
significance for tiie other (essay 2. F = 2.405, p = .0603). Furthermore, the Index of 
Writing Awareness continues to be a useful instrument for predicting student writing 
ability and correlates with high statistical significance to such distinct dependent 
\^ariables as writing quality and the length of compositions. Thus, based upon the 
results, computer-guiaed writing software, as defined and designed in this study, has a 
limited, but statistically significant effect on novice writers. 
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CHAPTER V 

DISCUSSION 

Outline of the Chapter 

Three main sections comprise this chapter: addressing the research questions 
that framed this study; analyzing the theoretical and practical implications of the 
results; and offering recommendations for future research. 

Computer-Guided Writing Software 
Promotes Rhetorical Effectiveness 

The first research question asked: "Does computer-guided writing software 
help students produce more rhetorically effective compositions?" This question was 
studied by comparing essays written by students using different versions of computer- 
guided writing software. The writings were rated using the California Assessment 
Program*s Scoring Guide for Rhetorical Effectiveness, a 6-point holistic rubric. 
Although the treatment that was hypothesized to have the greatest positive impact 
(CGI) offered inconsistent results, the model featuring chunking and on-line guides 
(CG) regularly produced statistically significant differences when compared to the 
control group. Even though the students in the CG and full treatment (CGI) groups 
began the experiment with an identical 1.625 mean score on the writing pre-test, the 
former demonstrated consistent improvement over the treatments (adjusted means for 
essay 1 = 2.450 and essay 2 = 2.709). The fact that all groups of students received the 
same introductory lessons on the reflective essay and then were left to their software 
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for any additioiial instruction bodes well for computer-guided writing as a teaching 
tool. 

Distinguishing Types of Computer Guides 

The second research question was posed to begin defining the types and 
characteristics of computer guides. The question asked: "Are there discernible types 
of computer guidance that can be used to promote specific expert-like performances 
in novices?" These hypotheses were tested by comparing student performance among 
treatment groups where the addition of one guide-type served as the sole 
distinguishing feature. For example, chunking (C) was compared to the control 
program that offered the same prompts resident on one screen rather than divided 
across six. Similarly, on-line guides (CG) were compared to chunking (C) because 
the buttons for "examples," "ideas," and "strategies" were the only difference 
between the programs. These guide- types were measured using instruments related 
to the guide's predicted effect, therefore, chunking and guides were measured by CAP 
ratings, interventions by the Index of Writing Awareness, and the full treatment by 
aggregate data describing the characteristics of expert writers. These ancillary 
hypotheses failed to produce statistically significant differences between key 
treatments. Although the strategy for determining guide effectiveness was logical, the 
realities of a limited treatment period and student adjustment to the programs may 
have confounded the experiment beyond such fine measures of distinction. 

In order to present the limited insights gained from this experiment regarding 
types of computer guides, the different types are discussed singly. First, cognitive 
psychology's notion of schematic chunking, although not statistically significant, may 
indicate a trend when compared to the control group (Scheff6 p - value of .1639). 
Thus, although not statistically significant at the predetermined confidence level of 
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5%, that some movement occurred during a three-week treatment period recommends 
structural chunking for additional research. 

Rhetorical interventions present a different result. Those groups using the 
interventions (CI and CGI) consistently showed lower scores in both the CAP and 
IWA ratings than their companion treatments without interventions (i.e., C vs. CI, and 
CG vs. CGI). As noted in the literature review, effecting change in such a 
fundamental process as metacognition may be a lengthy endeavor. Likewise, 
cognitive dissonance was an expected result of the interventions because novice 
writers were confronted by ways of thinking at which they are typically 
inexperienced. However, lowering scores are not the desired outcome for this type of 
prompt, and thus, rhetorical interventions as presently configured should be 
abandoned as an instructional strategy. 

When on-line guides were directly compared to the treatment group lacking 
only the buttons for examples, ideas, and strategies (C), no statistically significant 
differences were recorded. However, because on-line guides were a key feature of 
the most effective treatment (CG) for both measures of rhetorical effectiveness and 
cognition, the guides offer a distinct, statistically significant advantage. The fact that 
differences between the two related programs (C and CG) were non-significant may 
be attributable to such reasons as a synergistic effect between chunking and guides 
when used in conjunction, the ability of students to generate their own ideas when in 
a chunked program, the limited treatment period, or the small number of subjects. 

The full treatment incorporating all three types of guidance (chunking, 
interventions, and on-line guides) seems to reflect the attributes of its component 
pans. For example, when comparing CAP means for treatment essays, the CGI 
groups always scored above the control, but below its companion program (CG) 
which lacked the negative effects of rhetorical interventions. 
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Computer Guides and Learner Ability 

The third research question asked: **What is the interaction between learner 
ability and computer support?" This question was addressed by comparing the 
performance of low ability students in the full treatment against those in the control. 
The data used to measure this performance were CAP ratings of treatment essays and 
Index of Writing Awareness scores. In all but essay 1, mean scores from the control 
group surpassed those of the full treatment group, thereby suggesting that this 
configuration of guided writing software had a general negative impact on low ability 
writers. Additional analyses showed that low and high ability students had a similar 
slope across the guided writing software, albeit with commensurate ranges in scores 
between the classifications. Because low ability students in the on-line guide 
treatment group (CG) outperformed their peers in other groups on both treatment 
essays, it might be suggested that low abilitj' students benefit from the optional 
guides. On the other hand, scores on essay 2 show both intervention programs (CI 
and CGI) ranking below the control's means, perhaps suggesting that low ability 
students are hindered when their abready stressed writing process is interrupted or 
complicated by external forces (interventions). In summary, because low ability 
students tend to epitomize the characteristics of novice writers, developing software 
to support them is critical. The broad brushstrokes offered by this study add little 
concrete evidence to this goal, but may point in areas that are more fruitfully studied. 

User Satisfaction and Guided Writing 

The last research question took into consideration students' feelings: **Is there 
a relationship between computer support and a student's level of satisfaction in using 
computers as a writing tool?" The approach taken to answer this question involved a 
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Likert, bi-polar questionnaire administered before and after the experimental 
treatment. Response, as measured by group means showed that students' attitudes 
changed little over the course of the study. Reexamination of the questionnaire 
(Appendix C) revealed that its general nature may not have been able to detect 
changes evoked by one specific software program. In retrospect, it would have been 
useful to ask learners specifically about the version of guided-writing software they 
used as opposed to computers in general. Also, because students at OTarrell leam in 
a technology-rich environment, their attitudes toward computers likely included the 
experiences they'd recently had writing branching stories and exploring an adventure 
game about the civil war. The fact that student attitudes did not decrease after using 
the guided v/riting program may speak to the depth of their opinions or that the 
guided writing may not have been an unpleasant experience. Thus, to conclude, the 
generality of the instrument and the confounding variables associated with a wealth of 
experiences using computers makes any clear distinctions about user satisfaction hazy 
at best. 

Implications of the Study 

The most important implication of this study relates to the statistically 
significant difference in rhetorical effectiveness gained by students who used 
computer-guided writing software to compose their reflective essays. Because 
students who wrote with the on-line guide (CG) program showed an advantage over 
those who did not, the practical implications seem clear: when presenting novice 
writers with a new task, a chunked and supportive environment can raise writing 
quality. This finding does not suggest that computers will replace composition 
teachers (that old taunt), but wouldn't every writing instructor like to read a first draft 
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that has benefited from extra thought? An apt quotation by writer John McPhee helps 
to put the benefit of using a computer-guided writing program into perspective. He 
wrote: "I want to get to get the structural problems out of the way first, so I can get to 
what matters more" (in Murray, 1982, p. 51). With the array of authorware available 
such as H:/perCard™, HyperStudio^, and LinkWay™, teachers can create their own 
focused guiding programs, emphasizing the traits, characteristics, and structures that 
they feel their students need. Thus, customized programs could serve the self- 
discovery goals of the expressionists, support classic rhetorical skills, and prompt 
creative problem- solving. Additionally, the correlation between word count and 
composition quality suggests that promoting fluency (which has long been the 
strategy when working with novice writers) can now also promote rhetorical 
effectiveness with no additional teacher instruction. The practical implications are 
rich with potential. 

Theoretical implications also abound now that a benefit has been shown for 
guided writing programs. First, the work that has been done with procedural 
facilitation can be integrated with the literature of computer-use. Such areas as 
learner- control, motivation, creativity, and user-interface unite the two disciplines. 
Additionally, employing computers as partners in research holds theoretical 
implications. Tracking students' actual use with retrieval systems can provide 
insights into a variety of authentic writing processes, especially as related to the 
stiategies of novice and expert writers. 

Also, the notion that novice writers labor under a cognitive load has been 
unintciitionally supported through the use of rhetorical interventions. Because low 
ability writers especially tend to have limited cognitive resources (Bereiter & 
Scardamalia, 1987), a prompting environment must be sensitive to the load its use 
places on the learner. Although the interventions did not seem to alter the 
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achievements of high ability students, low ability students fairly crumbled with the in- 
process interruptions. Another clear implication of this study is the confirmation that 
writing is a complex task that features many interrelated and mutually excluding 
imperatives. For instance, the need to intervene in encapsulated procedures conflicts 
with the imperative not to over-stress novice writers' cognitive reserves. 

Similarly, longitudinal studies could contribute to our understanding of the 
affective and metacognitive effects of computer-guided writing. This descriptive 
research, in turn, could yield practical implications for prescribing types of guides 
targeting student success. 

Another aspect of the study that bears both theoretical and practical 
implications is the further confirmation that the Index of Writing A.wareness is a 
useful tool for research on composition. The high correlation between the instniment 
and srudent v/riting ability not only makes it useful as a predictor of ability, but also 
as another post- test measure. Of particular practical importance to researchers is that 
the IWA is an accurate tool that also does not add to the threat of maturation as a 
written pre-test might. Thus, the IWA is one more dependable tool for researchers 
investigating the cognitive development of younger writers. A practical implication, 
besides being a more objectifiable measure, is that classroom teachers could use the 
IWA as an early diagnostic test to better inform and individualize their instruction. 

Recommendations for Future Research 

Flaws and Suggested Improvements 

As stated by Bereiter and Scardamalia in The Psychology of Written 
Composition (1987), one of the problems with short-term instructional interventions 
is that measures are unlikely to detect significant change. This was particularly true 
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in the current study as related to assessments of cognitive and attitudinal change. 
Compounding this inherent shortcoming was the limited access students using the 
guided-writing software had to a very busy and productive computer facility. With a 
treatment period of three weeks and less than one week between guided essays, 
students had little time to alter such ingrained characteristics as metacognition 
concerning their writing processes and their feelings about computers. Improvements 
in this area are twofold: fu*st, a longitudinal study would allow time for more 
fundamental changes in students to occur, and second, more precise instruments 
could be developed to assess the specific nature of the guiding experience. 

Given the available access to computers, several procedural shortcomings of 
the study became apparent. First, students could have benefited from a direct lesson 
on using their selected software. Although they are regular computer users, 
instruction as to why the programs were set up as they were and how to respond to the 
program's features would be of benefit educationally to students. Second, even 
though a reading assignment or revision of previous writings were suggested as 
activities for students who had finished, the routine of turning to computer games 
became the actual activity. Given the relationship discovered between word count 
and writing quality, the fact that many students "finished'' their writings early, then 
played "Wagon Train 1848" and 3-D Tic-Tac-Toe suggests that a less enticing post- 
writing environment could have encouraged better writing. Improving on this 
situation would take altering the climate of the lab. The post-assignment activities of 
revising and reading would need to be emphasized to become a part of the routine or 
risk confounding the treatments themselves. 

As for weaknesses within the software itself, two stand out. First, as was 
made clear by the student outcomes, rhetorical interventions actually served to hinder 
the students' understanding of their writing processes. Two particular reasons for this 
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may be possible. First, as mentioned earlier, if a novice writer is cognitively taxed, 
additional complications amplify the load. Second, observations of the students using 
the interventions revealed that the mechanism of the interaction (a popping-up dialog 
box), carried along with it some conventions of user-interface. Typically, when a 
dialog box appears with its "OK" and "Cancel'' buttons (see Figure 5), users click one 
or the other and go on without reading the message. As regular users, many stu- 'ents 
reacted to the intervention by simply "OK'ing" it away. Others sat for minutes trying 
to write a worthy answer, and then attempted to return to the passage that had been 
interrupted. Improvements might see students type in and organize the rhetorical 
goals they have for each chunked section as part of a pre-vvTiting activity. This 
solution may promote the expert's executive strategizing without taxing novice 
writers during the composing process itself. Promoting the global monitor in novice 
writers is a positive objective supported by the literature; the exact method of 
applying this goal to computerized guidance remains to be solved. 

A second weakness of the software relates to its content, rather than its 
interface. The prompt of the second guiding program related to the students' current 
in-class focus on world events and social problems. Although a worthy subject, 
previous practice had students determine the issue underlying these controversial 
topics and then discuss possible solutions. Analysis of differences between essay 1 
and 2 across ability levels revealed that low ability students tended to interpret the 
prompt for essay 2 in this manner and fell into a persuasive mode. Thus, the mean 
scores for essay 2 dropped for many low ability students. High ability students, 
conversely, tended to not be lured away from reflective writing. This flaw in the 
wording of the prompt does demonstrate the importance of clarity in content, and, in 
negative fashion, the customizability of the guided-writing programs. Possible 
improvements involve more extensive testing the software in prototype form before 
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using it as part of formal instruction. This would be particularly true when a program 
is prepared by a department, school, or district to be used with many students. 

Possible Research Questio ns for Future Studies 

This study has offered a definition of computer guided-writing software and 
subsequently demonstrated the program's significant effect in promoting writing 
quality. This study is only a beginning. Besides replicating the experiment and 
improving upon its shortcomings, other experiments should be conducted to add to an 
understanding of how computer guidance works. Below are several questions ready 
for further research. 

1. What is the relationshi p between using a computer- guided writing program and 
development of expert characteristics? 

Although an attempt was made to answer this question in the current study, 
inconclusive evidence makes one wonder whether computer-guided writing software 
acts as a training wheel or a crutch? More precisely, do students develop their own 
strategies through interaction with the software or do users rely on prompts and 
guides to prop themselves up? Differences between pre-test writings and treatment 
essays showed dramatic increases in scores, but a model founded on cognitive 
theories should show students* schemata gaining in complexity and sophistication. A 
longitudinal study with pre- and post- tests of the Index of Writing Awareness may add 
to knowledge in this area. 

2. How does the content and stvle of on-line guides affect student performance in 
terms of rhe torical effect iveness and attitude ? 

Because the model featuring on-line guides yielded a statistically significant 
effect in CAP scores, further research into the nature of computer-guided writing is 
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warranted. The content of the guides in the current study was patterned after the work 
of Bereiter and Scardamalia (1987) and strove for informality, clarity, and specificity. 
It would be interesting to pursue how the content of in-process guides impacts 
students. In other words, why did the CG groups write better? Was it that they 
gained a sense of audience? Were they "facilitated" to employ more sophisticated 
strategies? Did their creativity enjoy an on-line boost? This area offers a wide range 
for future research. 

3. What are the actual writing processes of novice and expert writers ? 

Through the use of retrieval or tracking systems, the way a student uses a 
guided- writing program can be recorded. Not only does this offer data to support 
other research questions, but by studying the "paths'' students take through their 
process, more accurate descriptions and corresponding theories might inform 
improved teaching strategies. 

Conclusion 

The objective of this study was to define, develop, and test computer-guided 
writing software. Although the study did not provide a complete definition, 
significant pieces of a model were identified. Particularly important were the findings 
that show an in-process benefit from using on-line guides and stiiictural chunking. 
Further research will want to both broaden and deepen this beginning understanding 
of computer-guided writing's effect on students and their compositions. Particularly, 
as the pace of technological advancements continues to increase, and as more students 
learn to write at the keyboard of a computer, it seems only natural to investigate 
options that guide people toward successful experiences with writing. 
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Evaluation Component 

1) What is the hai^dest part or writing for you? 

1 a. Using complex words and sentences. 

2 b. Organizing the paper in a general theme. 

0 c. There is never anything difficult about writing for me. 

2) What would help you become a better writer? 

1 a. If more people would help you when you write. 
0 b. Writing papers with shorter words or phrases, 

2 c. Making sure that others will understand what you write. 

3) What might help the average person with his or her writing? 

0 a. Having notepaper and pencils always available. 

1 b. Having access to questions that guide his/her writing. 

2 c. To know when to organize and when to extend his/her ideas. 

4) What is special about the first sentence or two in every paper you write? 

1 a. They use key words to inform the reader and capture his/her attention. 
0 b. The first few sentences are the most interesting. 

2 c. They often tell what the story or paragraph will be about. 

5) How are the last sentences of a paragraph or paper you write special? 

0 a. They are exciting and cover new topic areas. 

2 b. They summarize thoughts and ideas for the reader. 

1 c. They are harder to write and are sometimes neglected by the writer. 

Planning Component 

6) If your teacher gave you a hard topic to write about, what should you do? 

0 a. Write about an easier topic. 

1 b. Try to tell at least some of the main or basic ideas. 

2 c. Ask yourself questions to get at what you know and do not know. 

7) When you get a writing assignment, what is one of the first things you should think about? 

2 a. What is going to happen or be presented in your story or paper. 

0 b. How long the assignment must be. 

1 c. Whether you can think of any interesting or exciting sentences for this paper. 

8) If the teacher told you to write a story that would be easy for him/her to remember the general 

meaning, what would you do? 

2 a. Have introductory and summary sentences in each paragraph. 

1 b. Make it easy for him/her by writing one or two summary sentences. 
0 c. List as many details about the topic as you could possibly know. 

9) Before you start to write, what kind of plans should you make to help you to write better? 

0 a. A writer doesn't need plans, he/she can usually just start to write. 

1 b. You could choose a comfortable place for coming up with ideas and then begin. 

2 c. You could think about why you are writing about this and then ask yourself 

questions about the topic. 

10) If you are writing well, but were thinking ahead that you might not have enough information for 

one or two key issues, what might you do? 

2 a. Try to generate or create more information by asking yourself questions 

0 b. Ignore that part of the assignment. 

1 c. Try to cover yourself by writing more on other parts of the assignment. 



Note: from 'The Development Of Children's Writing Awareness and Performance Within a 
Generative/Evaluative Computerized Prompting Framework" by C. J. Bonk and T. H. Reynolds, 1990, April, 
presented at the Annual Meeting of the Am^-rican Educational Research Association, Boston, MA. 
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Emulation Component 

1 1) Why do you go back and rewrite or recopy things over again? 

0 a. Because it is good practice. 

2 b. Because other possible readers may not understand it. 

1 c. Because you want to check your spelling, grammaj, and sentence structure. 

12) Why would you ask yourself about the logic or quality of an idea before adding it into your paper? 

2 a. Because every idea must be fit into the paper's overall theme and flow. 

0 b. No reason; good ideas can be added anywhere in one's paper. 

1 c. Because ideas must have a purpose. 

13) Why does what you want to say in a story or paper ever change? 

2 a. Because at any time you can think of new or related ideas. 
1 b. Because you can come up with more to say. 

0 c. Because you didn't write enough for the teacher on the first draft. 

14) What do you do if you don't know what the writing assignment means or what to write about? 

1 a. Read the assignment over again for ideas. 

0 b. Write what you can about anything tliat might be related. 

2 c. Think about the purpose of the assignment or ask for help. 

15) Why is writing different from story telling? 

1 a. In writing, the sentences and paragraphs have to end somewhere. 

2 b. When talking to someone you can see if he/she understands, but in writing you may 

have to answer questions that might be asked and explain more. 

0 c. There is no difference, you say the same thing. 

Conditional Knowledge Component 

16) If you were told that a neighbor who works at the Los Angeles Times might publish your article on 
"how to improve any school's cafeteria plan" if it was good enough, what would you do? 

1 a. Have a friend or possibly a teacher read it over befcffe you send it in. 

0 b. Like any paper, you would send out your ideas written in the order in which you 
thought them. 

2 c. Be creative in coming up with ideas, but also be critical of whether they really fit into 

what the LA, Time's readers might want to hear about 

17) If you are writing a paper for a science or social studies class, what should you do in order to cover 

the important information and write well? 

2 a. Constantly ask yourself questions about the goal of the project and the reader's needs. 

0 b. Skip writing about the words and phrases you don't understand. 

1 c. Concentrate on important issues and try to do your best. 

18) If you are v/riting an article for a children's magazine, which would help you the most? 

1 a. Rev/riting and reworking the article as many times as possible. 

2 b. Talking about it with someone to make sure they also understood it. 

0 c. Making sure everything is spelled correctly. 

19) If you were writing a paper for a class project, what is a valuable first step? 

1 a. Go to the library for information. 

2 b. List your initial ideas and write brief notes about them. 
0 c. Skip the parts of the assignment you don't understand. 

20) Which of these is the best way to revise and change a paper or story? 

0 a. Concentrate on finding the "right words" and spelling everything correctly. 

1 b. Think about suggested changes your teachers or friends might make. 

2 c. Think about how paragraphs and sentences might be reordered and explained. 
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Appendix C: Reflective Essay Practice 
Teache/s Guide 

Anticipatory Activity 

Begin by asking questions and then probing students for their answers. If 
diverse views do not arise naturally from discussion, play the "devil's 
advocate'' and draw out the complexity of the situation. This will lead 
students towards an appreciation of complexity and the purpose for 
reflection. 

• What is the best way to teach children? Should children be allowed to 

follow their interests and to celebrate who they are? Or should students 
learn discipline and learn to serve their society? How should schools be 
set up for the best learning? What can we discover about the abstract 
theme of ''education" or "learning?" 

• Why are some places in the world very poor and others very rich? Should 

the rich share with the poor? What is that makes the rich poor? What can 
we say about the abstract theme of "fairness" or "duty?" 

• Who are the people who count on you? Who are the people you depend 

upon? Are these people important to you? Have you ever let them down 
or disappointed them? Why? What is it that wouldn't let you do what you 
knew you should do? What can we say about the abstract theme of 
"community" or "responsibility?" 

• What is a key belief of your people/family/ancestors/race? Why are some 

of these beliefs similar? Why are some very different? Are some right and 
others wrong? Who would get to decide this? What lesson might this 
teach us about the abstract theme of "diversity?" 

• For an all expense-paid, one week vacation anywhere in the world, would 

you: 

V step on a cockroach? 

V pull the wings off a butterfly? 

V kill a family pet with your bare hands? 

V murder someone you consider an enemy? 

V take someone you love off a life support system? 

What is the value of a life? Where do you draw the line between getting 
rid of pests, murder, and mercy killing? What does this difference say 
about the abstract theme of "killing?" 
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Teacher's Guide: Reflective Essay Introduction 
Introduction 

Use the HyperCard stacks "Essay Means 'to Tr/'' with an LCD and 
overhead projector. Begin with the history and background tour led by the 
father of essays, Michel de Montaigne. Montaigne lived in a time of great 
dissent in France. Religious groups were terrorizing the country. As a 
lawyer, justice, and, later, the mayor of Bordeaux, Montaigne learned that 
passionately held beliefs could be very dangerous. His goal, and the source 
of his wisdom, was to explore topics that many people either never thought 
about or took for granted (i.e., the education of children, thumbs, cripples, 
etc.). His driving question (which he had engraved in a medal he hung above 
his desk) was "que s?ais-je?" ("What do I know?"). His goal, then, was not to 
prove an opinion was right (this type of certainty was killing thousands or 
Protestants and Catholics), but to explore a topic so that he could see it from 
many different perspectives and thus appreciate the complexity and other 
people's viewpoints. 

Use the handout "Reflective Essay Practice" and guide the students 
through questions 1 - 4 on the Presentation Notes. Draw out some key 
aspects: 

1) Reflective essays involve discovery and exploration, not proving. 

2) Reflective essays use several different examples, quotations, or 
stories to explore the topic. 

3) What is at the heart of a reflective essay is an absti'action (truth, 
justice, love, friendship, etc.). This must be the topic of the essay, 
not what someone did, different feelings, or a set of unexplored 
examples. The abstraction and the questioning of it should be the 
main focus of the paper. 

4) Students may not be used to this type of thinking. Some helpful 
suggestions are to think of related sayings, opposite personal 
experiences, or exploring what life would be like if the abstract 
theme did not exist (i.e., life without love, life without friendship, 
life without fairness, etc.). Also, probing "Why" people do what 
they do will steer students away from arguing that something is 
good or bad/right or wrong. 

5) Students should strive for sincerity and honesty. Their real 
thoughts, feelings, and experiences will make the paper enjoyable 
to write and to read. It is like they are having a conversation with 
themselves. 
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Teacher's Guide: Reflective Essay Introduction 

Students can work in groups or alone to come up with three examples for 
question #5. The occasion is something thaf s happened in the world, an 
int:;resting saying/ lyric /quotation, or a confusing or intriguing personal 
experience. One suggestion is for students to focus on something that 
brought out strong emotions for them. When we are emotional, we usually 
feel powerful universal themes or abstractions. 

The purpose of reflection: to better understand a complex/ confusing 
emotion or idea (an abstract theme) by exploring it from all sides with 
concrete examples and probing questions. 

Lead the students through page 2 of the "Reflective Essay Practice.'' 

Sample Essays: Non-Example & Example. 

Read the first sample of ''Banana Popsicle." This is basically an 
autobiographical incident where the writer creates an engaging 
reminiscence of a personal experience and then shares the stor/s 
significance. This is a "non-example" of a reflective essay. There is little 
questioning or asking why. 

Next, read the longer version, which is a good example of a reflective 
essay. Highlight how the writer branches off from the opening (occasion) 
by bringing up a saying ('If you love something, set it free"). We can see 
that love, and particularly the needy kind, is the abstraction in this essay. 
Next, notice that the writer brings up a related experience in order to 
explore the abstraction (reflection). Finally, in the last two paragraphs is 
the writer's attempt to both broaden (universalize) the topic to people in 
general and then end by suggesting a new understanding or outlook (final 
awareness). 

Getting Ready to Write: Brainstorming 

Use the brainstorming section from the "Lef s Write It!" stack branch off 
the main menu of the HyperCard stack to help students generate ideas for 
their own essays. When looking at a brainstorming topic, ask students to 
offer abstractions and questions that could be explored in that topic. Do 
this long enough so that you feel they have an understanding that they 
will be exploring, not proving, and that they see a few directions to go in 
exploring any given abstract feeling or idea. 
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Teacher's Guide: Reflective Essay Introduction 
Assignment 

Recall an inaportant experience in your life. This experience should be one 
that confuses, bothers, or intrigues you, one that you want to explore to 
deepen your understanding of life. 

In-Class Writing 

Have students write their first reflective essay on the word processor. 
Give them as much time as they need, then print out their essays. 
Students should have their names on the papers. (Names will be taped 
over) 

Administer Pre-Tests 

Test conditions should be maintained so that the students' responses are 
truly their own. Students can write directly on the tests. 

• Computer satisfaction questionnaire 

• Index of Writing Awareness (IWA) 

Teacher's Outline 



Anticipatory Activity: Questions for reflection 

Introduction: "Essay Means 'to Try'" HyperCard Stack + Handout 

The purpose of reflection: Definition 

Sample Essays: Non-Example & Example 

Getting Ready to Write: Brainstorming with "Essay Means 'to Try"' 
Assignment: Recalling a personal occasion 

In-Class Writing: Wordprocessing a reflective essay (Writing Pre~Test) 

Administer Pre-Tests 

• Computer satisfaction questionnaire 

• Index of Writing Awareness (IWA) 

Groups Use TJte Writer's Guide Programs for Reflective Essay # 1 
Groups Use TJte Writer's Guide Programs for Reflective Essay # 2 

Administer Post-Tests 

• Computer satisfaction questionnaire 

• Indexof Writing Awareness (IWA) 



ERIC 



147 



137 

Reflective Essay Practice 

Student Handout 

Presentation Notes: 

1) Michel de Montaigne made a medal that had a question on it. What did it 
ask? 

2) What did the word "essais" originally mean? 

3) What are the main phases of the reflective writing process? 

4) What two different type of things do ''abstractions" tend to be? 

5) Name three occasions you could write about and a related abstraction: 
occasion: abstraction: 



occasion: abstraction: 



occasion: ^abstraction: 



6) What is the purpose of writing a reflective essay? 
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Reflective Essay Practice continued 

The following is a step-by-step activity to give you practice putting 
together a reflective essay. 

1) In order to start you must have an autobiographical incident that is 
confusing, bothersome, or intriguing to you. List an occasion you could 
write about 



2) What topic lies at the center of your autobiographical incident? 



3) Is the topic you listed in #2 an abstraction (something that we can't hold in our 
hands, something like a belief, value, emotion, or idea)? 

yes no 

4) What is an abstraction? Underline the abstractions, and {Bracket} 

concrete details. 

Baseballs Happiness 

Entertainment Video Games 

a Guitar Music 

If you got more than two wrong, practice more before going on. 

5) If your answer to # 3 was "No" change your answer to #2 so that you are 
focusing on an abstraction. Your abstraction: 

6) Brainstorm tliree (3) options/opportunities for reflection: 

• A related quote/ saying/ song lyric: 

• An opposite personal experience: 

• A real life situation that shows how your abstract topic is 

important to all different kinds of people. 
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APPENDIX D 

SAMPLE SCREENS FROM "ESSAY MEANS 'TO TRY 
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ABSTRACT 
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ABSTRACT 



One hundred and forty-three seventh and eighth grade students were assigned 
by intact classes to one of five versions of computer-guided writing software. The 
programs were created combining three hypothesized types of guides: structural 
chunking, rhetorical interventions, and on-line guides. The different versions of 
software were used to test the three guide-types. During a treatment period of three 
weeks, students wrote two reflective essays using their assigned version of the 
software. The essays were scored holistically based upon the Califomia Assessment 
Program's Scoring Guide for Rhetorical Effectiveness. Students also completed a 
metacognitive assessment (the Index of Writing Awareness) and an attitudinal 
questionnaire concerning satisfaction in using computers. 

Analysis of the data showed support for one version of the guided-writing 
softwai^e. The program featuring structural chunking and on-line guides provided 
statistically significant differences compared to the control group in terms of 
rhetorical effectiveness. It also demonstrated a statistically significant benefit in 
cognition when compared to one of the other treatment groups. No significant 
differences in attitude toward using computers could be attributed to the treatments. 
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